All growing companies come to a point when their manual entry processes just can’t keep up. With data increasing at unprecedented rates, security measures are often too difficult to enforce.
It’s a very common problem, and recently, we worked with a company who needed to find an alternative to their manual data entry methods that were at risk of failing audits. Once their process was automated, they were able to provide an auditable workflow of information, something they never had before. They were also able to elevate the people who previously entered data to do more meaningful work.
If your company is still doing manual entry, you’ve probably at least seen an article or two about how great automated entry is supposed to be. Automated data extraction better preserves data integrity—it’s supposed to be more accurate. When we compare manual data entry to automated, though, is it really any better?
Manual entry: Outdated and irrelevant or a trustworthy tradition?
Does manual entry affect data integrity?
The short answer is yes.
Data entry can be nearly 100 percent accurate when it is double keyed or verified within professional data entry applications. Accuracy rates for double keyed data vary from 99.963 to 99.995 percent. For single keyed data, the accuracy rate drops to 96 percent.
The figures for double entry seem great when you’re working with small amounts of data. But if your company works with large amounts of data, you can expect anywhere from 0.5 to 4 errors per 10,000 entries with double entry—and that’s assuming the data entry clerk is having a good day. The error rate for data only entered once is a staggering 400 per 10,000 entries—significantly affecting even small amounts of data.
How else is manual entry challenging?
A simple Google search produces over 400,000 results that talk about the drawbacks of manual entry. Even looking for manual entry benefits brings more articles about its drawbacks than its benefits.
Let’s take a look at some of the top drawbacks of manual entry.
- Speed: In order for data integrity to be as high as possible, manual data entry needs to be double entered. Double entry takes twice as much time and twice as many people, however, so it doubles the time of single entry. Generally, a single data entry speed runs at around 10,000 to 15,000 keystrokes per hour.
- Ambiguity: Source documents can be unclear, with unlabeled fields or fields missing values. This leaves data entry clerks to make their best guess as to what should be entered. And there often isn’t a way to make sure these values are left out when the data is analyzed.
- Volume:Data influx is often unpredictable. Even if your company has a good system that maximizes speed and minimizes opportunities for ambiguity, a sudden spike in customer responsiveness could throw a wrench into your system.
- Productivity: Okay, so say your company has been doing manual entry for forty years. You probably have a system that knows how to manage speed, ambiguity, and fluctuations. Who’s performing your data entry, though? Is it an outside company, or are there employees devoting portions of their weeks to data entry? If it’s the latter, data entry is likely distracting employees from their core tasks, reducing their potential productivity levels and costing your company money.
Why do people still use manual entry?
With explosive advances in software development, articles from 2000 that touted the advantages of manual data entry cite very different advantages than current articles do.
Today, many benefits seem like defensive responses to people pushing for automated extraction.
- Catch oddities: Even if automated entry is faster than manual, there are certain situations where speed is not most important. If your company’s data is extremely complex, there simply might not yet be an application that can read or extract it accurately. Having a high degree of analysis tied into your data entry process means manual entry may be the best option. Additionally, by assigning data entry to the employees who created the data or are familiar with it, your company can be better assured that oddities will not be entered.
- Save training time: If your employees are already trained for and accustomed to manual data entry, trying to implement an automated application can be an unwanted hassle. End user adoption is a common difficulty to face when you implement new applications or processes. The training time and frustration that switching to automated entry will bring simply might not be worth the hassle.
- Identical results: When data entry is double entered, it returns results identically accurate to automated extraction. In fact, if the data needing to be entered is exceedingly complex or unstructured, there’s a good chance manual entry could be more accurate.
Automated data extraction: How much can we trust to computers?
Does automated extraction solve the integrity risks of manual entry?
If you’re new to this automated versus manual debate, you might be surprised to find that automated extraction can produce errors just like manual entry. No process is 100 percent error-free.
Typically, automated data entry gives an accuracy rate of 99.959 to 99.99 percent. Those statistics should look familiar to you—they’re virtually identical to the accuracy rates of manual double entry. Keep in mind that these high accuracy rates quickly decrease as your information needs to be read more—say if the data is handwritten and requires intelligent character or word recognition.
If you only consider the accuracy rates of automated extraction versus manual entry, it may seem easier to stick with your established mode of manual entry.
As you compare one option to another, though, keep in mind that automated applications do not need to be perfect—they just need to be better than the manual process they’re replacing.
How does automated data extraction fall short?
No technology is perfect. Let’s take a look at the four biggest problems with automated entry.
- Learning curve: For companies with many data entry clerks or employees who perform data entry as one of their many responsibilities, the very idea of training that many people to use an automated process may be overwhelming.
- Limitations: Developers who are accustomed to manually coding procedures to dictate data entry may feel limited by the implementation of an automated application. In this situation, your company may pursue an open-source product, so developers retain more control of the process.
- Nonstandard content: Sure, automated extraction has come a long way since the early 2000s. Still, it can be less accurate with complex, nonstandard documents. Perhaps your company has to deal with paper documents that have cursive handwriting. Or maybe you have to enter and then analyze hundreds of paper surveys with varying fields.The company we mentioned working with earlier had to deal with nonstandard documents. Each month, they received variable reports from a third party. In order to get our software to automate their process, we needed to build a preprocessor first. Preprocessors act as intermediaries between the original content and the automated application. With the help of a preprocessor, applications can digest complicated documents, producing meaningful data quickly.So, even if your company handles very complicated documents, don’t despair. Because of applications such as preprocessors, the automating application that you find does not necessarily need to be able to handle your documents as is.
- Inaccuracy: If your data only requires pattern matching derived from metadata, the accuracy rates for automated extraction hold at nearly perfect. But if your data does need character recognition—either from paper documents or PDFs—accuracy will suffer. Machines simply aren’t yet great at reading large text blocks of unstructured text.
Why should I trust an application over my employees?
Okay, so now that you know the good, the bad, and the ugly of manual entry, and you know some drawbacks of automated extraction, let’s find out why some people are so passionate about automated extraction.
- Cost and time: Let’s face it: people get bored. An automated extraction application can reduce employee boredom. Better than that, though, automated applications are faster than manual entry. That will save your company time and money—without reducing staff. Staff who have had to perform data entry as one of many responsibilities will be free to devote more time to more important processes.This was the case with the company we mentioned working with at the beginning of this post. The employee who was doing data entry as one of many responsibilities saves over a week’s worth of time every year—freeing her to do more meaningful work. Can you imagine how she felt when she found out she would never again have to do the mind-numbing entry and could focus on more meaningful work.
- Traceability: Information Governance, e-discovery, audits—it’s difficult to make sure all of them are being satisfied by manual entry because much of the manual process is undocumented and thus untraceable. Automated extraction can record a visible trail of the data extraction process. The company we recently worked with has been able to avoid further audit problems with their process automated.
- Security: In industries like banking or healthcare, confidentiality is king. Even having employees view the data to enter it into a database can be considered a liability and breach of confidentiality. When the process is automated, confidential data is no longer consistently exposed to humans. Increased security has been a great added benefit that the company we recently worked with now experiences every day.
How can I apply this to my company?
- Find one workflow that still relies on manual data entry. You don’t have to automate everything all at once. In fact, some workflows might need to remain manual, at least for now.
- Look at the data being handled and consider if it can be automated. And don’t take the first “No” you get. Applications that can handle nonstandard and unstructured data are being developed and improved constantly.
- Understand the workflow. If the workflow can and should be automated, write up where the data comes from, what is entered, and where the data is entered. This will help when discussing automation and looking for the right application.
- Find the right tool to extract and enter the data. It can be hard to sort through all the options, so remember to keep your company’s needs at the forefront throughout the search. (This is actually a huge part of what we do at OPIN. Contact us today to chat!)
- Find a higher value task for your employee to perform. Automating a process does not have to displace employees. Instead, find ways to maximize their knowledge by finding higher value tasks to fill their time that has been freed by automation.