Sensible Node SDK

Welcome! Sensible is a developer-first platform for extracting structured data from documents, for example, business forms in PDF format. Use Sensible to build document-automation features into your SaaS products. Sensible is highly configurable: you can get simple data in minutes by leveraging GPT-4 and other large-language models (LLMs), or you can tackle complex and idiosyncratic document formatting with Sensible's powerful layout-based document primitives.

This open-source Sensible SDK offers convenient access to the Sensible API. Use this SDK to:

• Extract: Extract structured data from your custom documents. Configure the extractions for a set of similar documents, or document type, in the Sensible app or Sensible API, then run extractions for documents of the type with this SDK.
• Classify: Classify documents by the types you define, for example, bank statements or tax forms. Use classification to determine which documents to extract prior to calling a Sensible extraction endpoint, or route each document in a system of record.

• For extraction and classification response schemas, see the Sensible API reference.
• For configuring document extractions, see SenseML reference.

• The latest version of this SDK is v0.
• The latest version of the Sensible API is v0.

• This SDK supports all non-end-of-life Node versions.
• This SDK supports all non-end-of-life Typescript versions.

In an environment with Node installed, open a command prompt and enter the following commands to create a test project:

mkdir sensible-test
cd sensible-test
touch index.mjs

Then install the SDK:

npm install sensible-api

Get an account at sensible.so if you don't have one already.

To initialize the SDK, paste the following code into your index.mjs file and replace *YOUR_API_KEY* with your API key:

// if you paste in your key, like `SensibleSDK("1ac34b14")` then secure it in production
const sensible = new SensibleSDK(YOUR_API_KEY);

Note: Secure your API key in production, for example as a GitHub secret.

To extract data from a sample document at a URL:

1. Install the Sensible SDK using the steps in the previous section.
2. Paste the following code into an empty index.mjs file:

import { SensibleSDK } from "sensible-api";

// if you paste in your key, like `SensibleSDK("1ac34b14")` then secure it in production
const sensible = new SensibleSDK(YOUR_API_KEY); 
const request = await sensible.extract({
      url: "https://github.com/sensible-hq/sensible-docs/raw/main/readme-sync/assets/v0/pdfs/contract.pdf",
      documentType: "sensible_instruct_basics",
      environment: "development"
    });
const results = await sensible.waitFor(request); // polls every 5 seconds. Optional if you configure a webhook
console.log(results);

2. Replace *YOUR_API_KEY* with your API key.
3. In a command prompt in the same directory as your index.mjs file, run the code with the following command:

node index.mjs

The code extracts data from an example document (contract.pdf) using an example document type (sensible_instruct_basics) and an example extraction configuration.

You should see the following extracted document text in the parsed_document object in the logged response:

{
  "purchase_price": {
    "source": "$400,000",
    "value": 400000,
    "unit": "$",
    "type": "currency"
  },
  "street_address": {
    "value": "1234 ABC COURT City of SALT LAKE CITY County of Salt Lake -\nState of Utah, Zip 84108",
    "type": "address"
  }
}

Navigate to the example in the SenseML editor to see how the extraction you just ran works in the Sensible app. You can add more fields to the left pane to extract more data:

You can use this SDK to extract data from a document, as specified by the extraction configurations and document types defined in your Sensible account.

See the following steps for an overview of the SDK's workflow for document data extraction. Every method returns a chainable promise:

1. Instantiate an SDK object with new SensibleSDK().
2. Request a document extraction with sensible.extract(). Use the following required parameters:
   1. (required) Specify the document from which to extract data using the url, path, or file parameter.
   2. (required) Specify the user-defined document type or types using the documentType or documentTypes parameter.
3. Wait for the results. Use sensible.waitFor(), or use a webhook.
4. Optionally convert extractions to an Excel file with generateExcel().
5. Consume the data.

You can configure options for document data extraction:

const request = await sensible.extract({
      path: ("./1040_john_doe.pdf"),
      documentType: "tax_forms",
      configurationName: "1040_2021",
      environment: "development",
      documentName="1040_john_doe.pdf",
      webhook: {
         url:"YOUR_WEBHOOK_URL",
         payload: "additional info, for example, a UUID for verification",
    }});

See the following table for information about configuration options:

Get extraction results by using a webhook or calling the Wait For method.

For the extraction results schema, see Extract data from a document and expand the 200 responses in the middle pane and the right pane to see the model and an example, respectively.

See the following code for an example of how to use the SDK for document extraction in your app.

The example:

1. Filters a directory to find the PDF files.
2. Extracts data from the PDF files using the extraction configurations in a bank_statements document type.
3. Logs the extracted document data JSON to the console.
4. Writes the extractions to an Excel file. The Generate Excel method takes an extraction or an array of extractions, and outputs an Excel file. For more information about the conversion process, see SenseML to spreadsheet reference.

import { promises as fs } from "fs";
import { SensibleSDK } from "sensible-api";
import got from "got";
const apiKey = process.env.SENSIBLE_API_KEY;
const sensible = new SensibleSDK(apiKey);
const dir = ABSOLUTE_PATH_TO_DOCUMENTS_DIR;
const files = (await fs.readdir(dir)).filter((file) => file.match(/\.pdf$/));
const extractions = await Promise.all(
  files.map(async (filename) => {
    const path = `${dir}/${filename}`;
    return sensible.extract({
      path,
      documentType: "bank_statements",
    });
  })
);
const results = await Promise.all(
  extractions.map((extraction) => sensible.waitFor(extraction))
);

console.log(extractions);
console.log(results);
const excel = await sensible.generateExcel(extractions);
console.log("Excel download URL:");
console.log(excel);
const excelFile = await got(excel.url);
await fs.writeFile(`${dir}/output.xlsx`, excelFile.rawBody);

You can use this SDK to classify a document by type, as specified by the document types defined in your Sensible account. For more information, see Classifying documents by type.

See the following steps for an overview of the SDK's workflow for document classification. Every method returns a chainable promise:

1. Instantiate an SDK object (new SensibleSDK().

2. Request a document classification (sensible.classify(). Specify the document to classify using the path or file parameter.

3. Poll for the result (sensible.waitFor().

4. Consume the data.

You can configure options for document data extraction:

import { SensibleSDK } from "sensible-api";

// if you paste in your key, like `SensibleSDK("1ac34b14")` then secure it in production
const sensible = new SensibleSDK(YOUR_API_KEY);
const request = await sensible.classify({
  path:"./boa_sample.pdf"
  });
const results = await sensible.waitFor(request);
console.log(results);

See the following table for information about configuration options:

Get results from this method by calling the Wait For method. For the classification results schema, see Classify document by type (sync) and expand the 200 responses in the middle pane and the right pane to see the model and an example, respectively.