Sturdy Statistics API Documentation (1.0.0)

Creates a new index. An index is the core data structure for storing data. Once the index is trained (see documentation for the train endpoint), an index may also be used to search, query, and analyze data. If an index with the provided name already exists, no index will be created and the metadata of that index will be returned.

Returns a list of all indices tied to your API key.

Returns all metadata belonging to the specified index.

Uploads documents to a temporary staging index for processing and/or storage. Documents are processed by the AI model if the index has been trained (see the train endpoint), and are stored if the parameter save=true.

Documents are provided as a list of dictionaries. The content of each document must be plain text and is provided under the required field doc. You may provide a unique document identifier under the optional field doc_id. If no doc_id is provided, we will create an identifier by hashing the contents of the document. Documents can be updated via an upsert mechanism that matches on doc_id. If doc_id is not provided and two docs have identical content, the most recently uploaded document will upsert the previously uploaded document.

Any additional fields in the document dictionary are stored within the index as metadata and will become available for training and querying tasks (see documentation for the train and query endpoints). While arbitrary metadata may be included and later queried, "binary" and "tag" data are required for supervised training of an index (no metadata is required for unsupervised training). See the documentation for label_field_names and tag_field_names in the train endpoint for more information about the required formats.

If the index has been trained, the response contains predictions produced by the trained AI model for each document. In order to obtain predictions without locking or mutating the index, set save=false.
If you wish to update an existing doc's metadata, you can do a shallow update by leaving the doc content field either empty or unchanged. The shallow update will the skip the model inference step and only update/append the metadata fields passed into the api. This provides a significant speedup to the api call.

Uploading docs is a locking operation. A client cannot call upload, train or commit while an upload is already in progress. Consequently, the operation is more efficient with batches of documents. The API supports a batch size of up to 1000 documents at a time. The larger the batch size, the more efficient the upload.

Nota Bene

Uploaded documents are saved in a staging index. The index is unaffected until a commit request is sent.

Queries the specified index. Queries are flexible, and may contain any combination of:

1. SQL conditionals applied to metadata stored in the index. See documentation for the filters parameter. Note that this option requires metadata to be stored in the index; see documentation for the upload endpoint.
2. Thematic grouping as identified by the trained AI model. You may filter by either a single topic group (topic_group_id) or by an arbitrary list of specific topics (topic_ids). Note that these options require the index to have been trained. You may filter by either topic_ids or by topic_group_id but not both simultaneously.
3. Search. The query may include a search term to filter documents. The search is semantic, based on inferred thematic content, but terms enclosed in double quotes will impose an exact match filter. See documentation for the query parameter.

The API returns a dictionary containing three different entries. Each entry provides a distinct view into how the data in the index matches your query, with increasing degrees of abstraction:

1. A ranked list of document objects. This provides a direct list of documents matching the query. By default these documents are sorted by relevance, but you may optionally specify any metadata field to sort by (see documentation for the sort_by parameter). Each result contains a short excerpt from the document that evinces your query; see documentation for the summarize_by parameter for information about this excerpt. Each result also contains all prediction values associated with the document, all metadata fields associated with the document, and an ordered list of the topics associated with the document.
2. A ranked list of topic objects. This thematic information is abstracted from the documents in the index, but still provides a granular summary of the content present in the data which matches the query. These topic objects are sorted by prevalence (i.e., % occurrence), not relevance (i.e., degree of match to the query). Each topic has a topic_id that can be used for additional downstream queries (see documentation for the topic_ids parameter).
3. A ranked list of topic group objects. This provides the highest-level overview of the content in the index which matches the query. These are also sorted by prevalence. Each topic group has a topic_group_id that can be used for additional downstream queries (see documentation for the topic_group_id parameter).

Loads a specified set of docs from the index. Supports a comma separated list of up to 500 doc_ids at a time. Additionally, the api provides the ability to specify the summary style of the docs by providing either a comma separated list of topic_ids, a topic_group_id, or a search query. The api will extract the most thematically relevant section of the doc to return. The API returns a dictionary containing field docs that contains a list of document objects in the order specified in the query.

This api allows you to directly run arbitrary SQL queries against you index's metadata. The availab fields include doc_id as well as any metadata you uploaded to your documents.

Permanently applies all changes made to the staging index to the production index. Note that upload saves new documents only to the staging index; a commit is necessary to use those documents for querying or for training. This is a locking operation: no data can be uploaded, trained, or committed while a commit is in progress.

Reverts all changes to the staging index back, and resets the staging index to match the state of the production index.

Create a deep copy of the selected index.

Create a deep copy of the selected index.

Trains an AI model on all documents in the production index. Once an index has been trained, documents are queryable (see documentation for the query endpoint), and the model automatically processes subsequently uploaded documents (see documentation for the upload endpoint).

The AI model identifies thematic information in documents, permitting semantic indexing and semantic search. It also enables quantitative analysis of, e.g., topic trends.

The AI model may optionally be supervised using metadata present in the index. Thematic decomposition of the data is not unique; supervision guides the model and aligns the identified topics to your intended application. Supervision also allows the model to make predictions.

Data for supervision may be supplied explicitly using the label_field_names parameter. Metadata field names listed in this parameter must each store data in a ternary true/false/unknown format. For convenience, supervision data may also be supplied in a sparse "tag" format using the tag_field_names parameter. Metadata field names listed in this parameter must contain a list of labels for each document. The document is considered "true" for each label listed; it is implicitly considered "false" for each label not listed. Consequently, the "tag" format does not allow for unknown labels. Any combination of label_field_names and tag_field_names may be supplied.

Returns most conditionally prevalent topics information.

Returns most conditionally prevalent topics information.

Returns the status of the current job.

List all jobs matching the filter criteria