AMAZON SAGEMAKER GROUND TRUTH

Through SageMaker Ground Truth, you can access the MTurk workforce and implement additional validation and quality checks for a scalable and cost-effective way to train and improve ML models. Use cases include data annotation and human data verification. SageMaker Ground Truth also offers curated workforces for Generative AI use cases including content generation, image captioning, human evaluation, prompt engineering, human feedback, and more.

Learn more and get started on the SageMaker Ground Truth website.

Developer Sandbox

A simulated environment for testing your tasks.

About the Sandbox

The Mechanical Turk Developer Sandbox is a simulated environment that lets you test your applications and tasks prior to publication in the marketplace.

Benefits:

Free to use for registered Mechanical Turk requesters. Fees will not be withdrawn and payments are not made to Worker accounts.
Has functional parity with the production website.
Requires only a URL change to configure your application to work against the developer sandbox or the production website.

Get Started

To access the sandbox, you will need a Mechanical Turk Requester account and, in order to access the sandbox programmatically, you will need an Amazon Web Services (AWS) account.

Requester Sandbox

Requester Site Sandbox

You can create test tasks in the Requester Sandbox to make sure your design or application works.

Visit the Requester Sandbox »

Worker Site Sandbox

You can view tasks created in the Requester Sandbox to see how they will appear to Workers.

Visit the Worker Sandbox »

Sandbox FAQs

How do I use the MTurk Developer Sandbox?

How do I configure my application to use the MTurk Developer Sandbox?

To use the MTurk Developer Sandbox during development and testing, set the URL service endpoint in your code to point to the sandbox endpoint: https://mturk-requester-sandbox.us-east-1.amazonaws.com

To use the MTurk Developer Sandbox for testing tasks using external questions, set the destination URL for the form action to be:

https://workersandbox.mturk.com/mturk/externalSubmit

To switch your application to the production MTurk system, set the URL service endpoint to: https://mturk-requester.us-east-1.amazonaws.com

To switch your tasks using external questions to the production MTurk system, set the destination URL for the form action to be:

https://www.mturk.com/mturk/externalSubmit

Example using Python:

import boto3

region_name = 'us-east-1'
aws_access_key_id = 'YOUR_ACCESS_ID'
aws_secret_access_key = 'YOUR_SECRET_KEY'

endpoint_url = 'https://mturk-requester-sandbox.us-east-1.amazonaws.com'

# Uncomment this line to use in production
# endpoint_url = 'https://mturk-requester.us-east-1.amazonaws.com'

client = boto3.client(
    'mturk',
    endpoint_url=endpoint_url,
    region_name=region_name,
    aws_access_key_id=aws_access_key_id,
    aws_secret_access_key=aws_secret_access_key,
)

# This will return $10,000.00 in the MTurk Developer Sandbox
print(client.get_account_balance()['AvailableBalance'])

Example for tasks using external questions:

Set the form action URL to your desired environment:

<body>
  <!-- This POST method is posting to the sandbox worker site-->
  <form method="POST" action="https://workersandbox.mturk.com/mturk/externalSubmit">
  <!-- This POST method is posting to the production worker site-->
  <!--<form method="POST" action="https://www.mturk.com/mturk/externalSubmit">-->
  <input type="hidden" id="assignmentId" name="assignmentId" value="">

What features of MTurk are not supported in the MTurk Developer Sandbox?

What are the guidelines and policies for the MTurk Developer Sandbox?

Developer Forum

Have a question about the Application Program Interface (API) or Command Line Tools (CLTs)? Post your questions and comments in the developer forum.

Visit the Developer Forum »