Artificial Intelligence in LegalTech – Beware the Mechanical Turk

Artificial Intelligence in LegalTech - Beware the Mechanical Turk
by Derek Karpel

Today, there is a tremendous push to transform the legal profession by using artificial intelligence and legal tech. So if you are a part of a general counsel or practice law, you know that legal tech is a hot topic. Quite often, we hear it in discussions about how we need to get more lawyers to automate their processes and embrace advanced predictive technologies.

Lawyers, of course, have no issue adopting the latest and greatest technologies within law firms and corporate general council offices. Still, to keep our industry relevant and pertinent, they need to make that happen sooner than later.

All machine learning algorithms are not built equally. This is where legal AI solutions step in. The general counsel must look at artificial intelligence in legal tech rather carefully. For example, when considering machine learning (ML) for contract management, they need to make sure that they are not purchasing a Mechanical Turk.


The Mechanical Turk was the first significant fraud loosely associated with artificial intelligence. In 1769, Wolfgang von Kempelen created what was touted as the world’s first automated master chess player. In the 1800s, he finally introduced it to North America, showcasing it at New York’s National Hotel and generating significant interest in the media. The hottest area in the development of AI in legal tech is in the contract management space.

The design, itself, included a life-size dummy seated in front of a large wooden cabinet, which was tightly packed with levers, gears, and cranks – similar to a mechanical clock. As a result, mechanical automation played a mean game of chess, completely amazing audiences across North America and Europe. It even played against famous opponents, such as American polymath Benjamin Franklin and French dictator Napoleon Bonaparte, who reportedly attempted a few illegal moves during a game.


In 1836, the machine was purchased by a group led by doctor John Kearsley Mitchell. And after spending several months examining it, the group unlocked the secret of the machine. It was controlled by a person who was hidden within the mechanism. Still, the charade lasted another 80 years, and in the end, the device was donated to Philadelphia’s Chinese Museum.


Do not confuse this famous fraud case with the Amazon Mechanical Turk, though. Increasingly, Amazon uses humans to automate, research, and perform small jobs for companies. They may soon provide more legal research and administrative work in our own industry, too.


While watching out for the snake-oil sellers and Mechanical Turks of the world, legal professionals need to consider adopting AI solutions. They hold intense promise for the future of the profession. And our system at ContractPodAi is already making big strides.


Contract management is the hottest area when it comes to artificial intelligence and legal tech. And ContractPodAi has tools that apply AI technologies to analytics, allowing for high-quality reviews of thousands of contracts. The machine learning element then comes back with statistics, highlights the inconsistencies, and brings forth all of the relevant data. All machine learning algorithms are not built equally.

The hottest area in the development of AI in legal tech is in the contract management space.

Given that the average Fortune 1000 Company has 20,000 to 40,000 active contracts at any given time, we need technology to help us. And the beauty of true machine learning is that the machine, itself, does not grow tired like a human being. It can review 40,000 contracts without a single cup of coffee and will remember that contract #459 is, in fact, identical to contract #30,922.

The use of true machine learning in the legal profession is a deeply exciting development. Certainly, if you invested in legal technology or even considering such investments, AI is part of that equation. Nevertheless, ML algorithms are not built equally.


A recent Forrester report on AI technologies in law put forward a model suggesting that not all AI capabilities are the same. In its model, firms were categorized into one of four levels: pre-kindergarten, grade 8, and senior high school, as well as university or PhD. Because some development houses are more advanced than others, you should look closely at the maturity level of the machine when considering a vendor for your legal technology.

Most importantly, don’t be afraid to ask questions. You need to scrutinize a vendor’s use of ML/AI. You need to confirm that it’s a computer system that will replicate its own results instead of outsourced people doing AI work in the background (i.e. Amazon Mechanical Turk discussed above).

Ultimately, in addition to tinkering, prodding, and testing, you need to remain open about the system’s capabilities while slowly building your confidence around them.


Finding the ‘cranks’ in the industry is important. And because of this, national law journals are filled with these types of cases. But you can avoid making a poor decision, and risking your career and reputation by discovering them early on.

ContractPodAi partnered with IBM Watson – the most widely recognized and trusted ML system today – to develop E:V®. Purchasing a contract management software system based on Watson means that you never have to worry about a Mechanical Turk. In fact, the machine learning in contract management that we offer is not only the best in the industry, it is at the PhD level of AI (from a Forrester perspective).

Lastly, when considering systems based on artificial intelligence in legal tech, be sure to ask yourself the right questions:

  • Is the ML system something a few coders hacked together as an experiment or is it industry recognized?
  • Are people doing the ‘intelligence’ work in the background, or is it truly a replicable system?
  • Is this a system that I can trust and on which I can stake my career?

Connect with us for more help on avoiding the Mechanical Turks!

Derek Karpel

Derek Karpel


Request a demo

Contact us today for your personalised demo.