Not every API request finishes when the HTTP response arrives.
Modern applications frequently start work that continues in the background.
A user might:
- upload a large file
- generate a report
- export thousands of records
- process a video
- create an AI-generated document
- import a large dataset
- trigger a data migration
- generate an invoice
- start a backup
- process a payment-related workflow
- run a long-running calculation
Instead of waiting for the operation to finish, the API may immediately return:
`202 Accepted`
The response effectively means:
The application accepted the request, but the requested work has not necessarily completed yet.
This creates an entirely different testing problem.
A normal API test might verify:
- request was accepted
- response status is correct
- response body contains expected fields
But that is not enough for an asynchronous workflow.
A job can return `202` successfully and then:
- never start
- remain stuck in processing
- fail after several minutes
- report the wrong status
- complete but produce incorrect output
- disappear before completion
- be processed twice
- time out incorrectly
- return stale progress
- expose another user's job
- allow unauthorized users to access the job
- leave partially created data behind
- cause the client to poll forever
The important question is therefore not:
"Did the API accept the request?"
It is:
"Did the entire asynchronous workflow eventually reach the correct state and produce the correct result?"
This guide explains how QA teams can test asynchronous jobs systematically.
---
What Is Async Job Testing?
Async job testing verifies systems where an API request starts work that completes separately from the original request.
A typical workflow looks like this:
1. Client submits a request.
2. Server validates the request.
3. Server creates a background job.
4. API returns `202 Accepted`.
5. Response provides a job identifier or status URL.
6. Client checks job status.
7. Job progresses through states.
8. Job eventually succeeds or fails.
9. Client retrieves the final result.
10. System reaches the expected final state.
For example:
POST /reports/export
The server might return:
HTTP/1.1 202 Accepted
{
"jobId": "job-12345",
"status": "queued",
"statusUrl": "/jobs/job-12345"
}
The client then polls:
GET /jobs/job-12345
Possible responses:
{
"jobId": "job-12345",
"status": "processing",
"progress": 45
}
Eventually:
{
"jobId": "job-12345",
"status": "completed",
"progress": 100,
"resultUrl": "/exports/report-12345.csv"
}
Testing only the first `POST` would validate only the beginning of the workflow.
The real quality risk exists across the entire lifecycle.
---
Why Async Workflows Are Difficult to Test
Synchronous APIs are relatively straightforward.
You send a request and receive the result.
Async workflows introduce time and state.
Instead of:
Request → Response
you now have:
Request
↓
Job Created
↓
Queued
↓
Processing
↓
Completed
↓
Result Available
Or:
Request
↓
Job Created
↓
Processing
↓
Failed
That means QA must test both:
State correctness
and
Time-dependent behavior.
The application must not only produce the right state.
It must transition through states correctly.
---
The Async Job Lifecycle
Before writing test cases, identify the complete lifecycle of the job.
A typical lifecycle might be:
submitted
↓
queued
↓
processing
↓
completed
Failure paths may look like:
submitted
↓
queued
↓
processing
↓
failed
Other systems may support:
queued
↓
cancelled
or:
processing
↓
timed_out
or:
processing
↓
partially_completed
The exact states depend on the product.
QA should not assume that every system uses the same state model.
Document the actual contract.
For each state, identify:
- how the state is entered
- which transitions are allowed
- what response the API returns
- whether the state is terminal
- what the client should do
- whether retrying is allowed
- what data should exist
This state model becomes the foundation of the test strategy.
---
Start With the API Contract
Before automating async tests, understand what the API promises.
Document:
- initial endpoint
- HTTP method
- expected initial status code
- job ID format
- status endpoint
- result endpoint
- available job states
- progress behavior
- timeout behavior
- failure behavior
- retry behavior
- cancellation behavior
- authentication requirements
- authorization rules
- retention period
- error format
- duplicate request behavior
For example:
POST /exports
Expected:
202 Accepted
Response:
{
"jobId": "123",
"status": "queued",
"statusUrl": "/jobs/123"
}
Then:
GET /jobs/123
Possible states:
queued
processing
completed
failed
cancelled
The test suite should be built around this contract.
---
Test the Initial 202 Response
The first test should verify that the system correctly accepts asynchronous work.
Check:
- status code
- response body
- job identifier
- status information
- status URL
- required headers
- authentication behavior
- response schema
- job creation
Example:
POST /exports
Expected:
202 Accepted
And:
{
"jobId": "abc123",
"status": "queued"
}
Do not stop here.
The test should verify that the returned job actually exists.
For example:
GET /jobs/abc123
Expected:
200 OK
{
"jobId": "abc123",
"status": "queued"
}
A system that returns `202` but fails to create the corresponding job has a serious consistency defect.
---
Test Every Valid Job State
If the API supports:
- queued
- processing
- completed
- failed
- cancelled
then each state needs explicit testing.
A useful test matrix is:
| State | Expected Behavior |
|---|---|
| queued | Job exists and is waiting |
| processing | Work is actively progressing |
| completed | Result is available |
| failed | Failure is reported correctly |
| cancelled | Job stops and cancellation is reflected |
Do not test only the happy path.
The failure lifecycle is often more important.
---
Test State Transitions
It is not enough to verify that individual states exist.
Test whether the system moves between states correctly.
For example:
queued → processing → completed
should be valid.
But something like:
completed → processing
should normally be impossible unless the product explicitly supports reprocessing.
Create a state transition matrix.
Example:
| Current State | Next State | Should Be Allowed? |
|---|---|---|
| queued | processing | Yes |
| queued | failed | Possibly |
| queued | cancelled | Depends on product |
| processing | completed | Yes |
| processing | failed | Yes |
| processing | cancelled | Depends on product |
| completed | processing | Usually No |
| completed | failed | Usually No |
| failed | processing | Depends on retry design |
| cancelled | processing | Usually No |
This catches state-machine bugs that ordinary endpoint testing can miss.
---
Test Polling Behavior
Polling is one of the most important parts of an async workflow.
A client may repeatedly request:
GET /jobs/123
until the job reaches a terminal state.
A basic polling strategy might be:
queued
↓
wait
↓
processing
↓
wait
↓
processing
↓
wait
↓
completed
QA should verify that polling behaves correctly.
Test:
- polling starts after job creation
- polling uses the correct job ID
- polling stops when the job completes
- polling stops when the job fails
- polling stops when the job is cancelled
- polling does not continue forever
- polling handles temporary errors
- polling respects server guidance where applicable
- polling does not create excessive traffic
A successful job should eventually stop being polled.
---
Test the "Never Completes" Scenario
One of the most important negative cases is a job that never reaches a terminal state.
Imagine:
processing
processing
processing
processing
processing
...
If the client keeps polling forever, the application has a reliability problem.
Test what happens when:
- the job is stuck
- the worker disappears
- the queue stops processing
- the downstream dependency never responds
- the job remains processing beyond its expected duration
The client should have a defined strategy.
For example:
Maximum polling duration: 10 minutes
After that:
Job timed out waiting for completion.
The exact timeout depends on the product.
The important thing is that it must be defined and tested.
---
Test Client Polling Timeouts
There are two different timeout concepts that QA should distinguish.
Request timeout
How long the client waits for one HTTP request.
For example:
GET /jobs/123
might have a 10-second network timeout.
Workflow timeout
How long the client is willing to wait for the entire asynchronous operation.
For example:
Maximum workflow duration = 10 minutes
These are not the same thing.
A job can survive dozens of successful HTTP requests while still exceeding the overall workflow timeout.
Both should be tested independently.
---
Test Job Failure After Successful Submission
This is a critical scenario.
The initial request may succeed:
POST /reports
Response:
202 Accepted
But the job may later fail.
For example:
POST → 202
then:
queued → processing → failed
The client must not interpret `202` as final success.
Verify:
- failed status is returned
- error information is meaningful
- job stops processing
- result is not incorrectly marked available
- partial output is handled correctly
- user receives an appropriate message
- retry behavior is correct
A successful submission is not the same thing as successful completion.
---
Test Different Failure Types
Do not create only one generic "job failed" test.
Test different failure causes.
Examples:
Validation failure
The job input is invalid.
Dependency failure
A downstream service fails.
Timeout
The job takes too long.
Infrastructure failure
Worker or queue becomes unavailable.
Resource failure
The system runs out of memory, storage, or another required resource.
Permission failure
The job cannot access required data.
Partial data failure
Some records succeed while others fail.
Unexpected application failure
An internal exception occurs.
The final API behavior should be predictable in each case.
---
Test Partial Completion
This is a particularly important scenario for batch operations.
Imagine an import job processing 10,000 records.
Suppose:
- 9,700 succeed
- 300 fail
What should happen?
Possible product behavior:
completed_with_errors
or:
failed
or:
completed
with an error report.
QA should verify the actual product contract.
Check:
- successful records
- failed records
- error count
- error details
- final job status
- generated output
- retry behavior
- rollback behavior
Never assume that "completed" means every item succeeded.
---
Test Result Availability
A common async bug is:
status = completed
but:
result = unavailable
The job state and result availability must be consistent.
If the API says:
completed
then verify:
- result exists
- result can be accessed
- result belongs to the correct job
- result belongs to the correct user
- result is complete
- result is not corrupted
For an export:
status = completed
should normally mean that the generated file is actually ready.
---
Test Completion Before Polling
Another edge case occurs when a job completes extremely quickly.
For example:
POST
↓
202
↓
completed
before the client performs its first status request.
The polling logic must handle this correctly.
It should not assume that it must observe every intermediate state.
A valid sequence might be:
queued → completed
without the client ever observing:
processing
That does not necessarily mean the system is broken.
The test should validate the contract rather than assuming that every intermediate state must be visible to clients.
---
Test Duplicate Job Submission
What happens when the client sends the same request twice?
For example:
POST /exports
followed immediately by:
POST /exports
Possible expected behaviors include:
- create two jobs
- reject the duplicate
- reuse the existing job
- require an idempotency key
The answer depends on the product.
But QA must test it explicitly.
Duplicate jobs can create:
- duplicate invoices
- duplicate reports
- duplicate notifications
- duplicate imports
- duplicate payments
- unnecessary infrastructure usage
Do not assume that asynchronous processing automatically makes duplicate submissions safe.
---
Test Retry Behavior
Async systems often contain retries at multiple levels.
There may be:
- client retries
- API retries
- queue retries
- worker retries
- downstream service retries
This can create unexpected behavior.
For example:
Client
↓
API
↓
Queue
↓
Worker
↓
Payment Service
If the payment service times out after processing the transaction, the worker may retry.
That can potentially produce duplicate side effects unless the operation is designed safely.
QA should test:
- retry count
- retry delay
- retryable errors
- non-retryable errors
- final failure
- duplicate side effects
- retry exhaustion
---
Test Cancellation
If the application supports cancellation, test it as a complete workflow.
For example:
POST /reports
returns:
jobId = 123
Then:
POST /jobs/123/cancel
Verify:
- cancellation request is authorized
- job enters the correct state
- worker stops processing where possible
- result is not incorrectly marked completed
- partial output is handled correctly
- repeated cancellation behaves predictably
Also test cancellation races.
For example:
Client requests cancellation
↓
Job completes at almost the same time
Which state wins?
The product should have a defined answer.
---
Test Authorization on Job Status
This is easy to overlook.
Suppose:
User A creates job 123.
User B should not necessarily be able to access:
GET /jobs/123
Test:
- same user
- different user
- administrator
- unauthenticated user
- expired session
- invalid token
- revoked access
A job ID should not become an authorization bypass.
This is especially important because job identifiers are frequently exposed to frontend applications.
---
Test Result Authorization
Do not stop authorization testing at the job endpoint.
Test the actual result.
For example:
GET /jobs/123
might correctly reject User B.
But:
GET /exports/report-123.csv
might accidentally allow access.
The complete chain must be protected:
Job
↓
Status
↓
Result
↓
Download
Authorization should be validated at every externally accessible boundary.
---
Test Job ID Security
Job identifiers can sometimes reveal implementation details.
Check whether IDs are:
- predictable
- sequential
- guessable
- reusable
- exposed in URLs
- exposed in logs
- exposed in browser storage
A predictable ID is not automatically a vulnerability.
The real question is whether knowing another job ID allows unauthorized access.
Test authorization independently from ID complexity.
---
Test Expired Jobs
Some systems retain job information only for a limited period.
For example:
Job created
↓
completed
↓
retained for 24 hours
↓
expired
Test:
- status after expiration
- result after expiration
- expired job response
- user messaging
- cleanup behavior
- repeated access
Do not allow an expired job to appear permanently active.
---
Test Polling After Completion
Once the job is complete, what happens if the client continues polling?
Possible behavior:
200 OK
status = completed
or:
410 Gone
or another documented response.
The important point is consistency.
Test repeated requests after:
- completion
- failure
- cancellation
- expiration
This catches state lifecycle problems.
---
Test Polling Under Temporary Server Errors
Polling can encounter:
- 500
- 502
- 503
- 504
- network timeout
- connection reset
The client should distinguish between:
Job failure
and
Status-check failure.
For example:
GET /jobs/123 → 503
does not necessarily mean:
Job 123 failed.
It may simply mean:
The status service is temporarily unavailable.
The client should not incorrectly mark the underlying job as failed unless the API contract says so.
---
Test Backoff Behavior
A badly designed polling client might do:
GET
GET
GET
GET
GET
GET
GET
as quickly as possible.
This can create unnecessary load.
A better strategy may use increasing intervals:
1 second
2 seconds
4 seconds
8 seconds
...
The exact strategy depends on the application.
QA should verify:
- minimum polling interval
- maximum polling interval
- backoff behavior
- maximum polling duration
- behavior after temporary errors
- behavior after terminal states
---
Test Progress Reporting
Some async jobs expose progress:
{
"status": "processing",
"progress": 45
}
Progress itself needs testing.
Check:
- starts at the expected value
- increases correctly
- does not move backward unexpectedly
- reaches the expected final value
- does not report 100% before completion
- does not remain at 100% while the job is still processing
Also test jobs where progress cannot be calculated precisely.
The product may intentionally expose:
progress: null
or omit progress entirely.
Do not force artificial progress requirements onto systems that cannot accurately measure it.
---
Test Progress Accuracy
If the system reports percentage completion, verify that it reflects meaningful work.
For example, an import of 100 records should not necessarily report:
90%
after only 10 records.
The exact calculation depends on the product.
More importantly, verify that progress does not create false confidence.
A job showing:
100%
while the final database transaction is still running can create a serious UX problem.
---
Test Queue Delays
A job may be accepted but remain queued for a long time.
Test:
submitted
↓
queued for extended period
↓
processing
Verify:
- status remains accurate
- user receives appropriate information
- timeout rules are respected
- queue delays do not cause false failure
- monitoring captures unusually long queue times
This is especially important in systems where workers are shared across customers.
---
Test Worker Failure
Consider:
queued
↓
processing
↓
worker crashes
What happens next?
Possible expected behavior:
processing
↓
retry
↓
processing
↓
completed
Or:
processing
↓
failed
QA should test the documented behavior.
A job should not remain permanently stuck simply because the worker handling it disappeared.
---
Test Recovery After Infrastructure Failure
Simulate failures such as:
- worker restart
- queue restart
- temporary database failure
- dependency outage
- network interruption
Then verify whether the job:
- resumes
- retries
- fails safely
- gets duplicated
- gets lost
The most important question is:
What happens to the job state when infrastructure disappears in the middle of execution?
---
Test Exactly-Once vs At-Least-Once Processing
Distributed systems often make strong guarantees difficult.
A job system may provide:
At-most-once processing
The job should not execute more than once, but it may be lost.
At-least-once processing
The job should eventually execute, but it may execute more than once.
Exactly-once behavior
The system attempts to ensure the observable business effect happens exactly once.
QA should understand which guarantee the product actually provides.
Do not assume that a queue automatically provides exactly-once business behavior.
For operations with side effects, verify the business result rather than simply checking worker execution.
---
Test Data Consistency After Failure
Suppose a job performs:
1. create record
2. update inventory
3. generate invoice
4. send notification
Then step 3 fails.
What remains?
Potentially:
- record created
- inventory updated
- invoice missing
- notification not sent
Is that expected?
Maybe the system should roll everything back.
Maybe partial completion is acceptable.
Maybe the job should resume from step 3.
The answer depends on the product design.
QA should test the actual consistency guarantees.
---
Test Restart and Resume Behavior
Some long-running jobs support resume.
For example:
Job starts
↓
4,000 / 10,000 records processed
↓
worker crashes
↓
worker restarts
↓
job resumes
Verify:
- previously processed records are not incorrectly duplicated
- remaining work is processed
- final count is correct
- progress is correct
- failed records remain traceable
- job eventually reaches the correct terminal state
Resume functionality is particularly important for large batch operations.
---
Test Long-Running Jobs
Do not test only jobs that complete in a few seconds.
Create realistic scenarios where jobs take longer.
Examples:
- 1-minute job
- 5-minute job
- 30-minute job
- large dataset
- slow downstream service
Verify:
- status remains available
- authentication remains valid
- UI remains usable
- polling remains controlled
- progress remains accurate
- timeout rules behave correctly
- result remains accessible
Long-running behavior often reveals bugs that short test runs never expose.
---
Test Small, Medium, and Large Jobs
Create a test matrix based on workload size.
| Workload | Purpose |
|---|---|
| Empty | Validate zero-work behavior |
| Tiny | Fastest successful path |
| Small | Normal workflow |
| Medium | Typical production workload |
| Large | Resource and timeout behavior |
| Very large | Limits and failure handling |
A system that works with 10 records may behave very differently with 100,000.
---
Test Empty Jobs
An asynchronous workflow may legitimately have nothing to process.
For example:
Export users where status = "archived"
If there are zero matching records, what should happen?
Possible results:
- completed with an empty file
- completed with zero records
- failed because there is nothing to process
- rejected before job creation
Whatever the product contract says, test it explicitly.
---
Test Malformed Job IDs
Test:
GET /jobs/invalid
and variations such as:
- empty ID
- malformed ID
- extremely long ID
- special characters
- nonexistent ID
- expired ID
Verify that responses are:
- predictable
- safe
- appropriately authenticated
- not leaking internal information
---
Test Job Status Endpoint Under Load
The status endpoint itself can become a bottleneck.
Imagine thousands of clients polling every few seconds.
The application might technically process jobs correctly while the status API becomes overloaded.
Test:
- many simultaneous pollers
- many active jobs
- completed jobs
- failed jobs
- long-running jobs
- high polling frequency
Measure:
- response time
- error rate
- CPU
- database usage
- cache usage
- queue pressure
Async testing is not only about the worker.
The entire lifecycle needs to scale.
---
Test UI Behavior for Async Jobs
If a frontend uses the async API, test what users actually see.
Typical UI states include:
Starting...
Queued
Processing 35%
Processing 70%
Completed
Failed
Timed out
Verify that the UI:
- displays the correct state
- does not show success too early
- does not freeze indefinitely
- handles refresh
- handles browser navigation
- handles network interruption
- allows retry when appropriate
- allows cancellation when supported
- provides useful failure information
---
Test Browser Refresh
This is a simple but valuable test.
Start an async job.
Then:
refresh browser
What happens?
A weak implementation may lose the job state entirely.
A stronger implementation may restore the job from its ID.
Test:
- refresh
- close and reopen tab
- navigate away
- return to the page
- open the job URL directly
The job should remain associated with the correct user.
---
Test Multiple Jobs at the Same Time
Users often start more than one operation.
For example:
Export A
Export B
Export C
Verify that:
- each job has a unique identity
- statuses do not cross over
- progress belongs to the correct job
- results are not mixed
- cancellation affects only the intended job
This can expose state-management bugs in both frontend and backend systems.
---
Test Job Ordering
Some systems require jobs to execute in order.
Others allow complete parallelism.
Test scenarios such as:
Job A submitted
Job B submitted
Job C submitted
Verify whether the expected behavior is:
A → B → C
or:
A
B
C
in parallel.
Do not assume submission order equals execution order.
---
Test Dependency Between Jobs
Some systems create chained jobs.
For example:
Job A
↓
Job B
↓
Job C
Test:
- A succeeds
- A fails
- A times out
- B starts only after A succeeds
- C starts only after B succeeds
- failed dependencies stop downstream jobs appropriately
This becomes particularly important in data-processing and workflow systems.
---
Test Scheduled Async Jobs
Some applications combine scheduling with background processing.
For example:
Schedule job for 02:00
↓
Job starts
↓
Processing
↓
Completed
Test:
- scheduled time
- timezone behavior
- missed schedule
- duplicate schedule
- manual execution
- cancellation
- retry
- overlapping executions
Scheduling introduces another dimension of state and timing.
---
Test Async Jobs Across Deployments
A deployment can happen while jobs are running.
Test:
Job processing
↓
Application deployment
↓
New version starts
Verify whether jobs:
- continue
- pause and resume
- restart safely
- fail
- duplicate
- disappear
This is especially important for systems using containers, queues, workers, and rolling deployments.
---
Test Version Compatibility
Suppose a job is created by version 1 of the application.
Then version 2 is deployed.
The job eventually completes under version 2.
Can version 2 still process the old job format?
Test:
- old job + new worker
- new job + old worker where applicable
- old payload + new schema
- old result format
- migration during processing
Long-running jobs can outlive the application version that created them.
---
Test Database State Independently
Do not rely entirely on API responses.
If an async job is supposed to create:
10,000 records
verify the actual persisted state.
Check:
- record count
- relationships
- status fields
- timestamps
- duplicate records
- orphan records
- transaction consistency
The API might say:
completed
while the database contains incomplete work.
Backend verification is often essential for high-value async workflows.
---
Test Notifications Triggered by Completion
Some jobs send notifications when they finish.
For example:
Job completed
↓
Email sent
or:
Job failed
↓
Slack notification
Test:
- successful completion notification
- failure notification
- duplicate notification
- delayed notification
- notification content
- recipient authorization
- notification when job is cancelled
Do not assume that completing the job automatically means the notification workflow works.
---
Test Observability for Async Jobs
Async failures can be difficult to investigate.
A useful system should provide enough information to trace a job.
Capture where appropriate:
- job ID
- request ID
- user ID
- timestamps
- state transitions
- worker information
- retry count
- failure reason
- dependency failures
- final result
The job ID should ideally allow QA and developers to connect:
API request
↓
Job
↓
Worker
↓
Database changes
↓
Result
↓
Notification
Without this traceability, debugging asynchronous failures becomes much harder.
---
Common Async Testing Mistakes
Mistake 1: Testing only 202
A `202 Accepted` response does not prove that the job succeeded.
Test the entire lifecycle.
---
Mistake 2: Waiting a fixed amount of time
Avoid:
wait 30 seconds
assert completed
This can create slow and unreliable tests.
Prefer polling with:
- terminal states
- maximum duration
- controlled intervals
- clear timeout errors
---
Mistake 3: Assuming every state must be observed
A very fast job might transition:
queued → completed
before your test sees `processing`.
Test the contract, not an assumed sequence.
---
Mistake 4: Treating polling failure as job failure
A `503` from the status endpoint does not automatically mean the underlying job failed.
Separate infrastructure errors from job state.
---
Mistake 5: Ignoring duplicate submissions
Repeated requests can create duplicate work or side effects.
Test them explicitly.
---
Mistake 6: Testing only successful jobs
Production failures are part of the lifecycle.
Test:
- worker failures
- timeouts
- dependency failures
- cancellation
- partial completion
---
Mistake 7: Ignoring authorization on results
Protecting `/jobs/{id}` is not enough if `/results/{id}` is publicly accessible.
Test the entire chain.
---
Mistake 8: Using sleeps instead of conditions
A test like:
sleep(30)
is usually weaker than:
poll until completed or timeout
Condition-based waiting provides faster success and clearer failures.
---
A Practical Async Job Test Matrix
A reusable matrix can look like this:
| Test Area | Example |
|---|---|
| Submission | Valid request creates job |
| Response | Returns 202 and job ID |
| Job lookup | Job can be retrieved |
| Queue | Job remains queued correctly |
| Processing | Job enters processing state |
| Completion | Job reaches completed |
| Failure | Job reaches failed |
| Cancellation | Job can be cancelled |
| Timeout | Stuck job eventually times out |
| Polling | Client stops at terminal state |
| Backoff | Polling does not overwhelm API |
| Result | Completed result is available |
| Authorization | Other users cannot access job |
| Duplicate | Repeated submission behaves correctly |
| Retry | Retry policy works correctly |
| Partial completion | Partial results follow contract |
| Worker recovery | Job survives worker interruption |
| Large workload | Large jobs complete correctly |
| Refresh | UI restores job state |
| Multiple jobs | Job states remain isolated |
| Expiration | Old jobs behave correctly |
| Deployment | Jobs survive supported deployments |
| Observability | Job can be traced end-to-end |
---
How to Automate Async Job Testing
A good automation pattern is:
1. Submit job
2. Capture job ID
3. Validate initial response
4. Poll status endpoint
5. Stop on terminal state
6. Fail on timeout
7. Validate final state
8. Validate result
9. Validate side effects
Pseudo-code:
response = POST("/exports")
assert response.status == 202
job_id = response.body.jobId
deadline = current_time + MAX_WAIT
while current_time < deadline:
status = GET("/jobs/" + job_id)
if status.state == "completed":
break
if status.state == "failed":
fail("Async job failed")
if status.state == "cancelled":
fail("Async job was cancelled")
wait(POLL_INTERVAL)
else:
fail("Async job timed out")
result = GET("/jobs/" + job_id + "/result")
assert result is valid
The exact implementation will depend on the test framework.
The important design principle is:
Wait for a meaningful state, not an arbitrary amount of time.
---
Building Reliable Async Tests
Async tests can themselves become flaky if they are poorly designed.
Follow these practices.
Use deterministic test data
Create unique records for every test.
Avoid shared state.
Control the timeout
Every polling test should have a maximum duration.
Capture the final state
When a test fails, record:
- job ID
- current status
- elapsed time
- last response
- retry count
- error details
Preserve artifacts
Keep:
- API responses
- logs
- job status history
- result files
- database snapshots where appropriate
Separate workflow tests from load tests
A functional async test should verify correctness.
A load test should evaluate scale.
Do not turn every functional test into a massive concurrency experiment.
---
AI-Assisted Async Job Testing
Async workflows are also a useful area for AI-assisted testing.
An AI testing system can analyze an API specification and identify potential asynchronous workflows.
For example:
POST /exports
returns 202
response contains jobId
An AI system could infer:
- validate 202 response
- validate job ID
- poll status endpoint
- test queued state
- test processing state
- test completed state
- test failed state
- test timeout
- test duplicate submission
- test authorization
- validate final result
The more valuable use of AI is not simply generating more test cases.
It is identifying the relationships between:
Request
↓
Job
↓
State
↓
Result
↓
Side Effect
AI can also help identify missing transitions.
For example, if the API documentation defines:
queued
processing
completed
failed
but no test covers:
processing → failed
that may represent a useful coverage gap.
AI should assist QA rather than become the only source of validation.
---
Measuring Async Job Quality
Traditional test metrics such as pass/fail counts are useful but incomplete.
For asynchronous systems, consider tracking:
Job completion success rate
How many jobs eventually reach the expected terminal state?
Average completion time
How long do jobs normally take?
P95/P99 completion time
How long do the slowest important jobs take?
Queue wait time
How long do jobs remain queued before processing begins?
Failure rate
How frequently do jobs fail?
Retry rate
How often do jobs require retries?
Timeout rate
How often do jobs exceed the expected duration?
Duplicate processing rate
How often does the same business operation execute more than once?
These metrics can reveal problems that a simple green test suite may hide.
---
Async Testing in CI/CD
A practical CI strategy is to divide tests into layers.
Pull Request
Run fast deterministic tests:
- submission
- status lookup
- successful completion
- failure handling
- authorization
- basic timeout behavior
Integration Pipeline
Run:
- queue behavior
- worker recovery
- dependency failures
- cancellation
- retries
- result validation
Nightly
Run heavier scenarios:
- large workloads
- long-running jobs
- multiple concurrent jobs
- infrastructure interruptions
- extended polling
- recovery scenarios
Pre-Release
Run critical production-like workflows:
- deployment during active jobs
- large data processing
- dependency outages
- worker restarts
- complete end-to-end lifecycle
Not every async scenario belongs in every CI stage.
The goal is useful feedback without turning the pipeline into a multi-hour test.
---
Example: Testing a Report Export Workflow
Consider:
POST /reports/export
The API returns:
202 Accepted
{
"jobId": "R123"
}
The client polls:
GET /jobs/R123
Eventually:
{
"status": "completed",
"resultUrl": "/reports/R123.csv"
}
A strong test should verify:
Step 1
Submit export request.
Step 2
Verify:
202 Accepted
Step 3
Capture `jobId`.
Step 4
Verify the job exists.
Step 5
Poll until completion or timeout.
Step 6
Verify terminal status.
Step 7
Download the result.
Step 8
Validate:
- file exists
- correct format
- correct number of records
- correct data
- correct user
- no duplicate records
- no missing records
Step 9
Repeat with an invalid request.
Step 10
Repeat with a large dataset.
Step 11
Test duplicate submission.
Step 12
Test authorization with another user.
That is an actual asynchronous workflow test.
Not simply:
assert 202 == response.status
---
Async Job Testing Checklist
Use this checklist when testing any background job system.
Submission
- [ ] Valid request creates a job
- [ ] Invalid request is rejected correctly
- [ ] Correct HTTP status is returned
- [ ] Job ID is returned
- [ ] Job status information is returned
- [ ] Job can be retrieved
Lifecycle
- [ ] Queued state works
- [ ] Processing state works
- [ ] Completed state works
- [ ] Failed state works
- [ ] Cancelled state works if supported
- [ ] Invalid state transitions are prevented
Polling
- [ ] Polling uses the correct job
- [ ] Polling stops after success
- [ ] Polling stops after failure
- [ ] Polling stops after cancellation
- [ ] Polling has a maximum duration
- [ ] Polling handles temporary API errors
- [ ] Polling uses appropriate intervals
Results
- [ ] Result exists after completion
- [ ] Result belongs to the correct job
- [ ] Result belongs to the correct user
- [ ] Result is complete
- [ ] Result is not available prematurely
Failure
- [ ] Worker failure is handled
- [ ] Dependency failure is handled
- [ ] Timeout is handled
- [ ] Retry behavior works
- [ ] Partial completion follows the contract
- [ ] Failed jobs do not remain stuck indefinitely
Security
- [ ] Job authorization is enforced
- [ ] Result authorization is enforced
- [ ] Invalid job IDs are handled safely
- [ ] Users cannot access another user's jobs
- [ ] Users cannot access another user's results
Reliability
- [ ] Duplicate submissions behave correctly
- [ ] Worker restart is handled
- [ ] Queue interruption is handled
- [ ] Deployment during processing is handled
- [ ] Long-running jobs work correctly
- [ ] Large workloads work correctly
---
Final Thoughts
Asynchronous workflows change what it means for an API request to succeed.
A `202 Accepted` response is only the beginning.
The real test starts after the response.
QA needs to validate:
Request
↓
Job Creation
↓
Queue
↓
Processing
↓
State Transitions
↓
Completion or Failure
↓
Result
↓
Side Effects
The most dangerous async bugs are often not obvious API failures.
They are situations where:
- the job says completed but the result is incomplete
- the job remains processing forever
- a retry creates duplicate work
- a failed worker leaves the job stuck
- one user can access another user's job
- the result exists but contains partial data
- the client polls forever
- a deployment silently loses active jobs
That is why asynchronous testing should be treated as a lifecycle problem rather than a single API assertion.
The strongest async test suites verify not only whether a job starts, but whether it finishes correctly, fails safely, remains observable, and leaves the system in a consistent state.
For QA teams working with APIs, queues, background workers, exports, imports, AI processing, and distributed applications, async job testing can provide a level of confidence that ordinary request-response testing simply cannot.