Detect and Highlight Entities from Unstructured Text with Custom Models

After performing step 1 of the tutorial Detect and Highlight Entities from Unstructured Text with Pre-trained Models to access and authorize the Business Entity Recognition Swagger UI, use the POST /datasets endpoint to create a dataset that will be used to train a custom model to extract entities from the text.

1. Click the endpoint name to expand it.

2. Click Try it out.

   

3. In payload, enter a description for your dataset, "Tutorial dataset", for example. Leave the parameter datasetType as it is.

4. Click Execute.

   

5. Copy the datasetId from the Response body.

   

To see the details of your newly created dataset, use the GET /datasets/{datasetId} endpoint.

1. Click the endpoint name to expand it.

2. Click Try it out.

3. Enter the datasetId obtained in the previous step and click Execute.

   

You receive a response like below. Among the datasetId and the description, you see the number of documents. Those documents include your training data and you upload some in the following step.

To upload training documents to your dataset, use the POST /datasets/{datasetId}/documents. You may repeat this step to upload multiple documents.

Please bear in mind that Business Entity Recognition requires your data to be in a specific format to train a model. See data format prerequisites in Training Data Format.

As an alternative to uploading your own JSON file to the service, you can use the following sample files (right click on the link, then click Save link as to download the files locally):

• Sample Training Data 1

• Sample Training Data 2

Please repeat this step twice and upload one document each time. The more data is available the better predictions the model can give you.

To upload documents, do the following:

1. Click the POST /datasets/{datasetId}/documents endpoint name to expand it.

2. Click Try it out.

3. Enter the datasetId obtained in step 1 and select one training document for the upload.

4. Click Execute.

   

5. Copy the documentId from the Response body.

   

Using the datasetId obtained in the previous step, you can obtain the details of your training documents using the GET /datasets/{datasetId}/documents/{documentId} endpoint.

1. Click the endpoint name to expand it.

2. Click Try it out.

3. Enter the datasetId obtained in step 1.

4. Enter the documentId obtained in step 3.

5. Click Execute.

   

You should receive a response, with the document details, like below. This shows you the size of the document in bytes.

Once the training documents are uploaded, you can submit a training job. This triggers the training of a machine learning model based on your uploaded data. For that, use the POST /training/jobs endpoint.

1. Click the endpoint name to expand it.

2. Click Try it out.

3. In payload, enter a modelName for your custom model, "Tutorial_custom_model", for example, and the datasetId obtained in step 2.

   CAUTION:

   Be aware of the following modelName guidelines:​

   • You cannot use the same name as a Business Entity Recognition pre-trained model (sap_email_business_entity and sap_invoice_header)

   • You cannot use "sap_" as a prefix

   • Start with an alphanumeric character

   • You can include the special characters "-" and "_"

   • You can use a maximum of 64 characters

4. Click Execute.

   

5. Copy the jobId from the Response body. This allows you to check the status of the training.

   

This indicates that your training job has been successfully submitted.

To check whether your training already succeeded, you can use the GET /training/jobs/{jobId} endpoint to see the current status of the job.

1. Click the endpoint name to expand it.

2. Click Try it out.

3. Enter the jobId obtained in step 5.

4. Click Execute.

   

You should receive a response like below. The status RUNNING indicates that the training is still in progress. In case the status is PENDING, then the training has not started yet.

You may check the status now and then. Please note that the training may take up to 5 hours. Afterwards, the training succeeds and the status switches to SUCCEEDED. Along with that, you receive all the capabilities of the model, i.e. the things that the model can recognize.

In some cases it makes sense to delete the training job. For example, in the Trial version of the service, only one training job is allowed. See Trial Account Input Limits. Use the endpoint DELETE /training/jobs/{jobId} to delete a training job.

1. Click the endpoint name to expand it.

2. Click Try it out.

3. Enter the jobId obtained in step 6.

4. Click Execute.

   

This triggers the deletion of the training job.

To see all your models, you can use the GET /models endpoint to see the list of them.

1. Click the endpoint name to expand it.

2. Click Try it out.

3. Click Execute.

You should receive a response like the following:

The response includes the Business Entity Recognition pre-trained machine learning models (sap_email_business_entity and sap_invoice_header), and the new custom model you have created, in this case, Tutorial_custom_model.

Use the GET /models/{modelName}/versions endpoint to see the available versions and other details about a specific model.

1. Click the endpoint name to expand it.

2. Click Try it out.

3. Enter the modelName from step 5.

4. Click Execute.

   

You should receive a response like the following:

Finally, to make actual predictions, you have to deploy your model. You can do so by using the POST /deployments endpoint.

1. Click the endpoint name to expand it.

2. Click Try it out.

3. In payload, enter the modelName from step 5, and the modelVersion.

4. Click Execute.

   

5. Copy the deploymentId from the Response body to check the status of the deployment in the next step.

   

As with training jobs, you now have to check the status of the deployment every now and then. Use the GET /deployments/{deploymentId} endpoint to do so.

1. Click the endpoint name to expand it.

2. Click Try it out.

3. Enter the deploymentId obtained in step 9.

4. Click Execute.

   

You should receive a response like below. Please note that compared to training jobs deployments do not have a status SUCCEEDED. The status RUNNING rather indicates that the deployment is live and the model can be used for predictions.

Use the GET /inference/jobs/{jobId} endpoint to see the text extraction results and the confidence level of the Tutorial_custom_model custom model.

1. Click the endpoint name to expand it.

2. Click Try it out.

3. Enter the jobId obtained in the previous step and click Execute.

   

You should receive a response like the following:

In the response, you will find the prediction for the extracted entities. The prediction is made with a probability indicated by the confidence field which represents how certain the model is about the value provided. A confidence of 1 means that the model is 100% sure about its prediction. The model tries to provide a value for all its capabilities. Thus, you may see a different result depending on your model’s capabilities. In case the model cannot identify an entity in the text, it returns empty for it.

You have now successfully used your own custom model to get text entity predictions for the unstructured text you submitted to Business Entity Recognition.