Compatibility update — September 4, 2026. This article restores a workflow published on August 3, 2020. Amazon SageMaker A2I is no longer open to new customers, although existing customers can continue using it. The accompanying sample repository was archived on May 21, 2025. New SageMaker notebook instances now default to Amazon Linux 2023; Amazon Linux 2 notebook-instance support ended on June 30, 2026. Amazon Mechanical Turk, mentioned in the original workforce options, is scheduled to close on September 30, 2026. Existing A2I customers can still study or adapt the workflow, but new customers will need another system for human review. ([docs.aws.amazon.com](https://docs.aws.amazon.com/sagemaker/latest/dg/a2i-task-types-general.html))
Businesses use video and audio for advertising, customer service, media production, training, and education. As their media collections grow, accurate transcripts can support indexing, text search, organization, and accessibility.
Amazon Transcribe converts speech to text. General-purpose transcription can struggle with specialized names, acronyms, product names, and other domain terminology. Amazon Transcribe custom vocabularies let you supply terms that the service should recognize and format in a particular way.
This walkthrough demonstrates a human-in-the-loop feedback cycle: transcribe the media, select low-confidence words, send contextual clips to human reviewers through Amazon Augmented AI, extract validated technical terms, create a custom vocabulary, and transcribe the media again. The historical experiment then compares the original and revised transcripts using word error rate and term-level accuracy. The restored proof-of-concept notebook remains available in its archived repository. ([aws.amazon.com](https://aws.amazon.com/blogs/machine-learning/improving-speech-to-text-transcripts-from-amazon-transcribe-using-custom-vocabularies-and-amazon-augmented-ai/))
One representative error was the phrase “an EC2 instance,” which the baseline transcript rendered as “Annecy two instance.” After EC2 was included in the custom vocabulary, the example was transcribed correctly.
Solution overview
1. Perform a baseline transcription. Run Amazon Transcribe without a custom vocabulary and retain each recognized word, timestamp, and confidence score.
2. Route uncertain segments to people. Select words below a chosen confidence threshold. Instead of presenting isolated words, send reviewers a short media segment containing the word and its neighbors.
3. Build a vocabulary from reviewed corrections. Collect the human transcriptions, filter likely common English words, and manually validate the remaining domain-specific candidates.
4. Transcribe and evaluate again. Create an Amazon Transcribe custom vocabulary, run new transcription jobs with it, and compare the resulting transcripts with hand-produced references.
The portable principle is broader than these AWS services: confidence is useful as a triage signal, not as a correction by itself. Route uncertain contextual segments to people, convert validated corrections into domain constraints, and test those constraints on material that was not used to build them.
Historical prerequisites
The 2020 notebook expected an AWS account, an S3 bucket, a SageMaker notebook instance, an execution role, Python with Boto3 and NumPy, and a private review work team. The S3 input and output resources, workforce, and A2I workflow needed to be configured in compatible AWS Regions.
The article instructed users to attach the broad AWS-managed policies AmazonAugmentedAIFullAccess and AmazonTranscribeFullAccess, together with access to the relevant S3 objects. Those policies still exist, but they grant broad access. For a current implementation, review the required API calls and prefer a least-privilege customer-managed policy. ([docs.aws.amazon.com](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonAugmentedAIFullAccess.html?utm_source=openai))
The walkthrough used a private workforce and suggested adding yourself as a worker to preview the task interface. Current AWS documentation continues to describe private workforces managed through Amazon Cognito or an OpenID Connect identity provider, but A2I access is now limited to existing customers. ([docs.aws.amazon.com](https://docs.aws.amazon.com/sagemaker/latest/dg/a2i-get-started-api.html?utm_source=openai))
Sample media and test design
The notebook used four introductory Amazon SageMaker videos. Video 1, Fully-Managed Notebook Instances with Amazon SageMaker – a Deep Dive, supplied the baseline demonstration, sample human-review tasks, and part of the custom vocabulary. Video 2, Built-in Machine Learning Algorithms with Amazon SageMaker – a Deep Dive, tested the vocabulary.
Video 3, Bring Your Own Custom ML Models with Amazon SageMaker, supplied additional vocabulary terms. Video 4, Train Your ML Models Accurately with Amazon SageMaker, provided a second test.
Videos 1 and 3 were designated in-sample because their reviewed corrections contributed to the vocabulary. Videos 2 and 4 were out-of-sample: they did not contribute terms and were used to test whether the vocabulary transferred to related, previously unseen material. ([aws.amazon.com](https://aws.amazon.com/blogs/machine-learning/improving-speech-to-text-transcripts-from-amazon-transcribe-using-custom-vocabularies-and-amazon-augmented-ai/))
Step 1: Establish a baseline
The notebook starts a batch transcription job without a vocabulary and waits for the job status to become COMPLETED. It parses the resulting JSON into the complete transcript, sentence-level segments, word-level items, timestamps, and confidence scores.
The authors plotted the confidence-score distribution and selected a threshold of 0.4 for the first demonstration. In that archived run, 16 words fell below the threshold. The number is a property of that media file, service output, and historical run; it is not a recommended universal threshold.
Threshold selection creates a workload tradeoff. Raising the threshold can catch more transcription errors, but it also sends more correctly recognized or unimportant words to reviewers. Lowering it reduces human work while increasing the chance that relevant errors will be missed.
Step 2: Create the human-review workflow
The A2I portion has four resources or actions: a workforce, a worker task template, a flow definition, and one human loop for each submitted review task. The historical implementation used the SageMaker CreateHumanTaskUi and CreateFlowDefinition operations, followed by the A2I Runtime StartHumanLoop operation. A2I custom task types still use explicitly created human loops for existing customers. ([docs.aws.amazon.com](https://docs.aws.amazon.com/sagemaker/latest/dg/a2i-task-types-custom.html?utm_source=openai))
The worker interface presents a short section of the source video, the original machine transcript, instructions, a replay button, and a text area for the corrected transcription. Reviewers are told to preserve punctuation that belongs to technical terms and to hyphenate multiword technical phrases so they can later be treated as single vocabulary entries.
Restored worker-template fields
The supplied archive lost several Liquid interpolation expressions and omitted the text-entry control, making its copied template unusable. The accompanying notebook preserves the missing material.
Media source: the template reads the S3 path from {{ task.input.filePath | grant_read_access }}. Displayed context: it inserts {{ task.input.video_title }} and {{ task.input.original_words }}. Playback range: JavaScript sets the starting position from {{ task.input.start_time }} and pauses at {{ task.input.end_time }}. Submission: a crowd text-area named transcription collects the reviewer’s answer.
The playback controls intentionally restrict the reviewer to the selected section. The surrounding words provide enough context to correct phrases rather than forcing a reviewer to interpret a low-confidence word in isolation. ([raw.githubusercontent.com](https://raw.githubusercontent.com/aws-samples/amazon-a2i-sample-jupyter-notebooks/master/A2I-Video-Transcription-with-Amazon-Transcribe.ipynb))
Selecting contextual clips
For each low-confidence word, the notebook selects up to three neighboring words on either side and obtains the first and last timestamps in that window. It submits the media path, start time, end time, original words, and video title as the human-loop input.
The restored notebook contains an important detail missing from the article’s shortened loop: after creating a task, it advances the current index by the three-word margin plus one. This avoids immediately creating another substantially overlapping clip. That detail reconciles the reported 16 words below the threshold with the 15 tasks created in the demonstration. ([raw.githubusercontent.com](https://raw.githubusercontent.com/aws-samples/amazon-a2i-sample-jupyter-notebooks/master/A2I-Video-Transcription-with-Amazon-Transcribe.ipynb))
The example review segments included phrases such as “every version of Annecy two instance is,” “I started using Boto three,” and “That’s the python Asi que.” These examples show why the reviewer needs audio and neighboring words rather than only the uncertain token.
When reviewers finish, A2I writes their answers to the configured S3 output location. The notebook retrieves completed human-loop records, reads the output JSON, and extracts the submitted transcription value.
Step 3: Extract candidate vocabulary terms
The notebook normalizes the reviewed text, splits it into words, removes punctuation around entries, and compares the results with the Natural Language Toolkit English words corpus. Words absent from that corpus become candidate technical terms, subject to simple checks for plurals and possessives.
This is a heuristic, not an authoritative classifier. A dictionary can omit ordinary words, names, inflections, abbreviations, and newer terminology. The historical output therefore included useful candidates such as SageMaker, Boto3, EC2, ECR, EBS, S3, SDK, IAM, VPC, Jupyter, BlazingText, /opt/ml, and mars.R, but it also contained ordinary or conversational words. Manual review remains necessary.
The final historical vocabulary contained display and pronunciation guidance for terms including “machine learning,” Amazon, Boto3, T3, ECR, EBS, Jupyter, /opt/ml, S3, SDK, SageMaker, IAM, VPC, EC2, BlazingText, and the speaker name Sarab.
Create the custom vocabulary
The notebook writes a table with the columns Phrase, IPA, SoundsLike, and DisplayAs, uploads it to S3, and invokes Amazon Transcribe’s CreateVocabulary operation. It waits until the vocabulary state becomes READY before using it.
Amazon Transcribe still supports table-formatted custom vocabularies. Current documentation describes the same four fields, allows them in any order, requires a Phrase value, and uses hyphens rather than spaces for multiword phrase entries. A vocabulary uploaded to S3 must be processed with CreateVocabulary before it can be selected in a transcription request. ([docs.aws.amazon.com](https://docs.aws.amazon.com/en_en/transcribe/latest/dg/custom-vocabulary-create-table.html?utm_source=openai))
Step 4: Transcribe again
After the vocabulary becomes ready, the notebook starts new transcription jobs with the vocabulary name in the job settings. It parses the revised output and writes the improved transcripts to separate text files.
This separation matters for evaluation: the ground-truth transcript, original machine transcript, and vocabulary-assisted transcript must remain distinct. Otherwise it becomes easy to compare the wrong artifacts or inadvertently evaluate a transcript against itself.
Evaluation with word error rate
Word error rate is defined as (S + D + I) / N, where S is the number of substitutions, D the deletions, I the insertions, and N the number of words in the reference transcript. Lower values indicate fewer edit operations relative to the reference length.
The historical notebook used JiWER and normalized the transcripts by lowercasing, removing punctuation, reducing repeated whitespace, and converting the strings into word lists. Because normalization affects the measured WER, the same transformation must be applied to the reference and both machine transcripts.
Current dependency note: the notebook did not pin JiWER. JiWER 4.0.0 was released in 2025, and its current transformation API uses reference_transform, hypothesis_transform, and ReduceToListOfListOfWords. The 2020 calls and transform names should therefore be updated or run in a historically compatible pinned environment rather than copied unchanged. ([jitsi.github.io](https://jitsi.github.io/jiwer/usage/?utm_source=openai))
Reported WER results
In-sample video 1: WER fell from 5.18% to 2.62%, a reported relative reduction of 49.4%.
In-sample video 3: WER fell from 11.94% to 7.84%, a reported relative reduction of 34.4%.
Out-of-sample video 2: WER fell from 7.55% to 6.56%, a reported relative reduction of 13.1%.
Out-of-sample video 4: WER fell from 10.91% to 8.98%, a reported relative reduction of 17.6%.
These figures are the archived experiment’s outputs, calculated from the supplied ground-truth, baseline, and revised transcript files. They describe four SageMaker videos processed with the 2020 system; they should not be treated as a general performance guarantee for Amazon Transcribe or custom vocabularies. ([aws.amazon.com](https://aws.amazon.com/blogs/machine-learning/improving-speech-to-text-transcripts-from-amazon-transcribe-using-custom-vocabularies-and-amazon-augmented-ai/))
Technical-term results
Aggregate correctness for the selected technical terms improved from 20% to 100% in video 1, from 12% to 100% in video 3, from 19% to 100% in video 2, and from 12% to 95% in video 4. These correspond to reported gains of 80, 88, 81, and 83 percentage points.
The notebook’s video 4 total-count row is internally inconsistent: it prints three correct baseline mentions while also reporting 12%, and its individual rows sum to five correct mentions out of 43, which is approximately 12%. This restoration preserves the supported percentage while not repeating the inconsistent count. ([raw.githubusercontent.com](https://raw.githubusercontent.com/aws-samples/amazon-a2i-sample-jupyter-notebooks/master/A2I-Video-Transcription-with-Amazon-Transcribe.ipynb))
The comparison illustrates why aggregate WER alone can hide important failures. Common function words dominate ordinary speech, while a small number of product names or acronyms may determine whether a transcript is useful for technical search, indexing, or topic classification. A modest change in overall WER can accompany a large improvement in the terms that matter most to the application.
What the experiment establishes
The experiment supports a narrow but useful conclusion: for these four related videos, a vocabulary derived from human-reviewed low-confidence segments improved both overall WER and recognition of selected AWS and machine-learning terms. Improvements on the two out-of-sample videos suggest that the vocabulary transferred beyond the exact media used to construct it.
It does not establish that every low-confidence word is wrong, that every transcription error has low confidence, or that a vocabulary built from one subject will generalize to unrelated material. Thresholds, context windows, reviewer instructions, vocabulary selection, normalization, and test-set separation all remain design decisions that should be evaluated for the intended domain.
Cleaning up
To avoid continuing charges, remove resources that are no longer required. Depending on what was created, this can include transcription jobs and vocabularies, A2I human loops and flow definitions, worker resources, S3 objects or buckets, CloudWatch resources, and the SageMaker notebook instance. Confirm retention and audit requirements before deleting review results or ground-truth data.
Conclusion
This workflow turns transcription uncertainty into a repeatable improvement process. It uses confidence to locate likely trouble spots, sends short contextual clips to people, converts validated corrections into a domain vocabulary, and evaluates the result on both source and unseen media.
The specific A2I implementation is now historical and unavailable to new A2I customers, but the architecture remains portable. A current system can replace A2I with another review queue while retaining the core loop: baseline transcription, confidence-based sampling, contextual human correction, vocabulary construction, retranscription, and held-out evaluation.
Originally published by Jasper Huang and Talia Chopra on the AWS Machine Learning Blog on August 3, 2020. ([aws.amazon.com](https://aws.amazon.com/blogs/machine-learning/improving-speech-to-text-transcripts-from-amazon-transcribe-using-custom-vocabularies-and-amazon-augmented-ai/?utm_source=openai))
Responses