The Moving Goalpost: How Remotasks Redefines Performance on the Fly

There is a number that follows you everywhere on Remotasks. It hovers over your dashboard like a verdict. It decides which tasks you can see, how much you earn, and whether you keep working at all. Workers call it the accuracy score, the quality score, the performance score. The platform has used different terms over the years. What it has never done is explain, in plain language, how the score is calculated, why it drops without warning, or how you appeal when you think the system got it wrong.

For hundreds of thousands of workers across Southeast Asia, Africa, and Latin America, that number is not an abstract metric. It is the difference between paying rent and not paying rent. Understanding where it comes from is not a luxury. It is survival. And the people who most need to understand it are the ones who have been given the fewest tools to do so.

What Remotasks Is, and Why It Matters

Remotasks is the consumer-facing labor arm of Scale AI, the San Francisco startup that has become one of the most powerful middlemen in the global AI industry. Founded in 2017, the platform works with companies like GM Cruise, Lyft, and Google, and claims to have paid out over $15 million to its 240,000-plus taskers. The work itself is foundational to modern artificial intelligence: workers draw bounding boxes around pedestrians in autonomous vehicle footage, classify content, transcribe audio, and rank AI-generated responses to help tune large language models. Without this labor, the AI products that Silicon Valley sells to the world would not function.

According to Reuters¹, Scale AI, which runs Remotasks as a subsidiary, reported revenues of $870 million in 2024 and now counts Meta Platforms as a 49 percent owner. The company is no longer a scrappy startup. It is infrastructure. And Remotasks is how that infrastructure gets its raw material, the human judgment that no algorithm has yet been able to replace.

Remotasks² is available in over 90 countries, making it one of the most globally accessible microtask sites. That global reach is, by design, also a global labor arbitrage. A Bangla writing expert in the United States might earn $22 an hour for the same work that earns a worker in India $2 an hour. The geography of desperation is the business model.

The Score That Governs Everything

When you join Remotasks, you complete training modules and a safety quiz. You pass an assessment. You start doing tasks. And then, sooner or later, you notice the score.

Your quality score determines which tasks you can access. A few bad reviews can lock you out of higher-paying projects. The scoring system is not always transparent about why scores drop.

That last sentence deserves to sit for a moment. The scoring system is not always transparent. That is not a minor inconvenience. It is a structural flaw in a system that controls the livelihoods of hundreds of thousands of people in countries where a few dollars an hour is meaningful money.

While Remotasks claims that workers can earn between $15 to $20 per hour, actual user reports suggest earnings are closer to $2 to $3 per hour. This discrepancy can be attributed to the accuracy score system, which affects payment based on how well tasks are completed. New users may find it challenging to maintain a high accuracy score, especially when starting out.

The mechanics work like this: every task you complete is reviewed, either by automated systems or by Remotasks’ internal review teams in the United States. If a reviewer decides your work does not meet the standard, your score drops. A low score can restrict you from higher-paying work categories. A sufficiently low score, or a sudden catastrophic drop, can lock you out of the platform entirely. Mistakes are final and will lower your score. In short, there is no redo.

But here is the central problem. The standard against which your work is judged is not publicly visible, consistently explained, or reliably communicated. Workers describe submitting tasks they believe are correct and watching their score fall anyway. They open support tickets. They wait. The answer they get back, if they get one at all, rarely explains what went wrong with enough specificity to prevent it from happening again.

Users also report suspensions, sudden project removals, unclear quality checks, and weak appeal options.

“Workers Are Unclear Why Their Work Is Approved or Rejected”

The opacity is not accidental. It is architectural.

Labour processes are obfuscated and there are often no redress mechanisms. Once a project is completed, it goes through several reviews before it is evaluated by the Remotasks teams in the US, but workers are unclear why their work is approved or rejected, and whether they get paid in full, or sometimes, nothing at all.

I keep coming back to that phrase: no redress mechanisms. In any functional employment relationship, there is at minimum a process. A person can ask why they were marked down. They can make a case. They can see the rubric. What Remotasks has constructed is not that. It is a system where a score changes, the change affects your earning capacity, and the explanation you receive is either absent or vague enough to be useless.

Consider what Rest of World³ reported about Usama Ali, a Python coding tasker in Pakistan who had worked on the platform since August 2023. When the platform went dark for him, he initially thought it was an error or a glitch.

“I got an email stating that my account was ‘under review. After that, I was unable to access my account.”

When he refreshed the page, he was shown a message saying the platform was blocked. No score explanation. No appeal window. No specifics about what had triggered the review.

This pattern is not unique to country-level bans. It shows up in the everyday experience of workers whose scores quietly decline. On Trustpilot⁴, one reviewer described putting in months of effort only to find the platform’s scoring unforgiving and the support unresponsive:

“Never ever trust this platform, few months back I sent them application, I passed but instead of moving me to real work, they started giving tons of trainings, all training passed and after doing just… [the account was restricted].”

Tasks like content sorting into predefined buckets can be strict. One wrong category can lower your quality score. A single wrong categorization. And the category definitions, in many projects, are genuinely ambiguous. Workers are expected to interpret guidelines written in English by teams in San Francisco and apply them consistently to culturally specific content from their own countries. When their interpretation diverges from the reviewer’s, the worker loses points. They are rarely told precisely which task failed or exactly what the correct answer was.

The Fairwork Verdict: 1 Out of 10

The clearest outside verdict on this system came from Oxford.

The Fairwork project, based at the Oxford Internet Institute⁵ and the WZB Berlin Social Science Center, scored Scale AI and Remotasks just 1 out of 10 on fair labor practices in its 2023 Cloudwork Ratings report. The report assessed fifteen platforms across five principles: fair pay, fair conditions, fair contracts, fair management, and fair representation. None of the platforms reviewed scored more than 5. A score of 10 out of 10, researchers note, simply means that a company is complying with the bare minimum.

Remotasks scored 1. One point. Out of ten.

The Oxford Internet Institute specifically highlighted Scale AI for “obfuscating” its labor process. That word is important. Obfuscating means making deliberately obscure or unclear. The researchers were not saying the scoring system was merely confusing. They were saying it was structured to prevent workers from understanding what was happening to them.

Scale AI responded to the report by stating it had made “meaningful investments to improve the contributor experience” and argued Fairwork’s “evaluation remains overly subjective in key areas.” It added that Remotasks contributors are independent contractors who choose when and how much they work and are paid on a per-task basis, and that guardrails in place ensure earnings meet or exceed local minimum wage rates.

That response is worth unpacking. Calling workers independent contractors who choose when and how much they work, is technically accurate and strategically misleading. The contractor classification is precisely what allows the platform to avoid the disclosure requirements, appeal rights, and minimum standards that come with employment relationships. It is the same move Uber made, the same move DoorDash made, and it has the same effect: it places the entire risk of the system’s opacity onto the worker while the platform retains all the power.

The Global South Bears the Cost

The performance score’s opacity does not harm all workers equally. It falls hardest on workers in the Global South, people for whom the difference between a functioning account and a suspended one is not a minor inconvenience but a financial crisis.

The Washington Post reported that many Remotasks workers in the Philippines earn far below the minimum wage. The Philippines has more than two million people doing crowdwork, many of them through Remotasks. Dominic Ligot, a Filipino AI ethicist, called these new workplaces “digital sweatshops.” Government officials in the Philippines admitted they were alarmed but weren’t sure how to regulate the platform. Data annotation is an “informal sector,” said department head Ivan John Uy. “Regulatory protective mechanisms are not there.”

That absence of regulation matters when the performance score becomes a weapon. Workers also don’t have any effective avenues to complain and can simply be removed from the platform. The score drops, the work disappears, and there is no labor tribunal to petition, no ombudsman to call, no employment lawyer who can take the case because there is no employment relationship to litigate.

Payments are frequently delayed, reduced, or withheld, with little recourse for workers. On Remotasks’ internal messaging platform, notices about late or missing payments are common.

In March 2024, Business Daily⁶ reported that this dynamic reached its most brutal expression. Remotasks shut down entirely in Kenya, Nigeria, and Pakistan. Workers were greeted by a message saying that “we regret to inform you that at the moment we are unable to provide service in your location.” No warning. No transition period. No explanation of what would happen to pending earnings.

In March 2024, Scale would block Kenya wholesale as a country from Remotasks. Thousands of workers who depended on the platform for their livelihoods were stranded without job security or, in many cases, owed wages. Workers in multiple countries reported not receiving their final payments when contracts ended or when they were suspended from the platform.

Grace Mumo, a single mother of three in Nairobi who had been supporting her family through Remotasks since losing her job in 2020, was cut off on March 7 without any explanation. “As I speak, I am wondering what the children will have for dinner because I have no money,” she told Rest of World⁷.

That is not an abstraction about labor policy. That is a woman not knowing how to feed her children because an algorithm somewhere in San Francisco changed a number without telling her why.

The Architecture of Informational Asymmetry

Let me be direct about what this system actually is. Remotasks operates on a fundamental informational asymmetry. The platform knows exactly why your score dropped. It knows which task reviewer marked you down. It knows the specific criteria your work failed. It has the data. What it has chosen not to do is share that data with you in a way that would let you understand and improve.

This is not a technical limitation. The data exists. Showing it to workers would cost almost nothing. What it would cost is control.

When workers understand exactly what the platform values and does not value, they gain bargaining power. They can document unfair reviews. They can organize around shared grievances. They can identify patterns of arbitrary or discriminatory scoring. Opacity prevents all of that. It keeps each worker isolated, blaming themselves for score drops they cannot diagnose, spending hours trying to second-guess a rubric they have never seen.

Common account issues include unclear quality checks and weak appeal options. In some cases, workers worry about pending earnings after losing access. That worry is rational. Workers who lose access to the platform mid-project have, in documented cases, lost the payment for the work they already completed. According to 60 Minutes⁸, there are documented cases where workers in Kenya say that sometimes before payday, the website shut them out of their accounts. They didn’t receive the pay.

The Medium⁹ investigation by the Relational Democracy Project describes what the Oxford Internet Institute noticed: Scale AI was named one of the Oxford Internet Institute’s top violators of fair labor practices for “obfuscating” its labor process. The researcher who spent five months working on Outlier, the platform Remotasks merged into in 2024, described thousands of Tier 3 contributors, professional experts in their fields, waiting without work in virtual lines behind a backlog of lost tasks, missing or specious reviews.

Missing or specious reviews. Reviews that workers cannot see, cannot understand, and cannot contest. That is the score system in action.

What Workers Are Actually Saying

Across forums, Trustpilot, Reddit, and Hacker News, the pattern in worker complaints is remarkably consistent. It is not that the work is hard. It is that the rules governing evaluation are invisible, inconsistent, and unappealable.

On Trustpilot¹⁰, a reviewer described the experience this way:

“One of the worst platforms to work on. Poor pay, late response and abuse by those who administer and give the training. It’s basically a scam for those who put their heart into remote work.”

Another reviewer wrote:

“They are running a company called Remotasks which is taking advantage of Africans and underpaying them severely. For tasks that they will pay $50 USD per hour in the US and India, they are paying $0.002 USD per hour.”

That reviewer’s frustration is with pay rates, but underneath it is the same structural issue: workers have no visibility into how the platform values their labor, no ability to negotiate, and no recourse when they are treated unfairly.

The aigigjobs.com review¹¹ is particularly candid: Reviewers can reject your work after completion, meaning you spent time on a task but do not get paid. Rejection rates vary by project and can be frustrating, especially when the guidelines are ambiguous.

Ambiguous guidelines. Rejections with no explanation. A score that controls your access to work and your earning potential, with no meaningful mechanism to challenge it. This is not an accident. This is a designed system, and the design serves the platform, not the worker.

The Broader Gig Economy Problem

Remotasks is not unique in using opaque scoring. Amazon Mechanical Turk has faced the same criticisms for years. Upwork’s Job Success Score baffles and frustrates freelancers who watch their scores fluctuate without clear explanation. Fiverr’s¹² Pro system has generated its own forums full of workers asking why their success scores dropped for reasons they cannot identify.

According to Time Magazine¹³, a survey of 752 workers in 94 countries found that all the platforms reviewed failed to meet even a basic threshold of labor rights standards.

But Remotasks occupies a particular position in this ecosystem. It is not just a freelance marketplace. It is the labor infrastructure of some of the most powerful and most funded AI companies in the world. The outputs of Remotasks workers go into products used by hundreds of millions of people globally. The quality of that work, including the accuracy of the scoring that determines whether workers are retained or discarded, affects the quality of the AI that shapes modern life.

There is a cruel irony here worth naming. Remotasks workers help train AI systems to be accurate, consistent, and fair. They spend their days making careful judgments about quality and correctness. And in exchange, they are subjected to a quality judgment system that is none of those things: not accurate in its explanations, not consistent in its application, and not fair in its consequences.

What Transparency Would Actually Look Like

This is not a hard problem to solve technically. The solution is straightforward, even if the will to implement it is absent.

Transparency would mean showing workers, at the task level, which submissions were marked down and exactly why. It would mean publishing the rubrics that reviewers use, in full, in the languages workers speak. It would mean providing a genuine appeal process with human review, not just a support ticket that closes after two weeks without a substantive response. It would mean showing workers their score trajectory over time with enough granularity that they can identify patterns rather than just receiving a final number.

It would also mean paying workers for tasks that are under review rather than withholding payment pending a review process the worker cannot monitor or influence. If a task is going to be rejected, the worker deserves to know why and deserves to know that before the payment window closes.

None of this is technically complex. It requires a different set of values: a belief that the people doing the work deserve to understand the rules of the system they are operating inside.

The Oxford Internet Institute’s Fairwork project¹⁴ found that Remotasks scored just 1 out of 10 on fair labour practices, failing on key metrics including its ability to fully pay workers. A score of 1 out of 10. On minimum standards. While operating at a valuation of billions of dollars, serving some of the most profitable companies in human history.

In May 2024, former Remotasks workers wrote directly to the White House. The letter alleged that workers in Kenya, Nigeria, and Pakistan were abruptly cut off without notice and owed substantial amounts of unpaid wages. CBS Detroit¹⁵ posted on how the workers from Kenya were being exploited by by US AI companies.

Joan Kinyua, a member of the group, stated that when Remotasks went offline, it pulled the rug from under them and depleted their means of living. Scale AI, based in San Francisco, largely escaped accountability.

That appeal went nowhere visible. Scale AI is headquartered in California. Its workers are classified as independent contractors in countries with limited gig economy regulation. The company’s legal exposure is minimal. Its reputational exposure, if the mainstream technology press decided to treat this story with the seriousness it deserves, is considerably higher.

Remote workers often have few reliable ways to contact supervisors or escalate complaints, despite the presence of hotlines and Slack channels. The existence of a Slack channel is not a redress mechanism. It is a gesture at transparency that provides the appearance of access while delivering very little of the substance.

The Closing Calculation

Here is what I keep returning to. Scale AI’s revenue in 2024 was $870 million. The company is backed by venture capital, valued in the billions, and now partly owned by Meta. The cost of building a genuine, transparent performance evaluation system, one that shows workers which tasks failed and why, that publishes rubrics, that provides real appeals, would be a rounding error in that revenue.

The performance score opacity is a choice. It is a choice about who the system is designed to serve. Right now, it serves the platform. It keeps workers in a state of uncertainty that makes them compliant, that suppresses collective action, that prevents the kind of organized pressure that might lead to higher pay or better working conditions.

The people doing the work are in Cagayan de Oro and Nairobi and Lahore. They are sitting in internet cafes and on the floors of small apartments with slow connections, drawing bounding boxes around pedestrians in California traffic footage, rating AI-generated responses for coherence and accuracy, making the careful human judgments that billion-dollar models depend on. And when a number on a screen decides they have fallen short, nobody will tell them how or why.

That is not a technical limitation. It is a decision. And the decision has a cost, measured not in dollars but in dinners not bought and rents not paid and children who go to bed hungry while a startup in San Francisco counts its billions.

The black box that runs your career does not have to be black. Someone just decided it should be.

Sources

1. “Reuters.Com” www.reuters.com/business/meta-pay-nearly-15-billion-scale-ai-stake-information-reports-2025-06-10/. Accessed 10 July 2026. ↩︎
2. “Remotasks” Earn $USD Doing Online Tasks from Home, 21 Apr. 2021, www.remotasks.com/en. Accessed 10 July 2026. ↩︎
3. Brandom, Russell. “Scale AI’s Remotasks platform is dropping whole countries without explanation” Rest of World, 28 Mar. 2024, restofworld.org/2024/scale-ai-remotasks-banned-workers/. Accessed 10 July 2026. ↩︎
4. Trustpilot, www.trustpilot.com/review/remotasks.com?page=4. Accessed 10 Sept. 2026. ↩︎
5. “OII | New Oxford Report Sheds Light on Labour Malpractices in the Remote Work and AI Booms” New Oxford Report Sheds Light on Labour Malpractic, www.oii.ox.ac.uk/new-oxford-report-sheds-light-on-labour-malpractices-in-the-remote-work-and-ai-booms/. Accessed 10 Sept. 2026. ↩︎
6. Kimuyu, Hilary. “Online gig site Remotasks exits Kenya” Business Daily, 13 Mar. 2024, www.businessdailyafrica.com/bd/corporate/technology/online-gig-site-remotasks-exits-kenya-4555340. Accessed 10 Sept. 2026. ↩︎
7. Brandom, Russell. “Scale AI’s Remotasks platform is dropping whole countries without explanation” Rest of World, 28 Mar. 2024, restofworld.org/2024/scale-ai-remotasks-banned-workers/. Accessed 10 Sept. 2026. ↩︎
8. 60 Minutes, www.facebook.com/60minutes/videos/ai-training-company-failed-to-pay-workers-say-former-employees/907657601330605/. Accessed 11 Sept. 2026. ↩︎
9. “Scale-AI’s Predatory Labor Practices” Medium, 20 May 2024, relationaldemocracy.medium.com/an-authoritarian-workplace-culture-4ba5f3666f9f. Accessed 8 July 2026. ↩︎
10. Trustpilot, www.trustpilot.com/review/remotasks.com?page=2. Accessed 10 Sept. 2026. ↩︎
11. Kim, Jordan. “Remotasks Review: Pay Rates, Task Types, and Is It Worth Your Time?” AI Gig Jobs, 3 Mar. 2026, www.aigigjobs.com/blog/remotasks-review-pay-rates. Accessed 8 July 2026. ↩︎
12. Fiverr, help.fiverr.com/hc/en-us/articles/36982827869457-A-Guide-to-Fiverr-Pro. Accessed 8 July 2026. ↩︎
13. Perrigo, Billy. “AI Gig Workers Face ‘Unfair Working Conditions,’ Study Says” 20 July 2023, time.com/6296196/ai-data-gig-workers/. Accessed 10 Sept. 2026. ↩︎
14. “OII | New Oxford Report Sheds Light on Labour Malpractices in the Remote Work and AI Booms” New Oxford Report Sheds Light on Labour Malpractic, www.oii.ox.ac.uk/new-oxford-report-sheds-light-on-labour-malpractices-in-the-remote-work-and-ai-booms/. Accessed 10 Sept. 2026. ↩︎
15. “CBS Detroit” Being overworked, underpaid, and ill-treated…, www.facebook.com/CBSDetroit/posts/being-overworked-underpaid-and-ill-treated-is-not-what-kenyan-workers-had-in-min/1119936310141559/. Accessed 11 Sept. 2026. ↩︎