0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *

87 − = 80

35 Ways Find Out if Youve Got Bad Data and Ways to Improve Validation Process

35 Ways to Find Out if You’ve Got Bad Data

Your data affects all aspects of your business, from marketing to product management. And bad data makes business rough.

The day-to-day decision making and management of your organization suffers if you have incorrect, misleading, or poorly formatted data in your databases,

This means inefficient operation. And it negatively impacts your bottom line.

But, bad data isn’t the source of all your problems, right?

It’s true. So, you must determine whether or not your business has a data problem. Otherwise you might end up chasing a red herring.

Fortunately, bad data has symptoms.

Here’s how bad data shows itself in each aspect of your business.

Real Estate

If you’ve got a data problem, your Agents, telemarketers, and operations staff can suffer from these problems:

  1. Trouble tracking owners for wholesale properties.
  2. Missing owner contact data such as a recent cell phone.
  3. No clue as to who owners are or where to find them.
  4. Buyers Leads lists get stagnant or outdated.
  5. Telemarketers and agents calling the wrong people and waste time.
  6. Little or no standard data management procedures.


If you’ve got a data problem, your marketing operation will suffer from these problems:

  1. Trouble tracking customer preferences and managing customer privacy.
  2. Missing data.
  3. No clue as to how much or what data is missing.
  4. Compliance problems.
  5. No way to perform data matching with internal or external files.
  6. Marketing cannot identify ideal customers.
  7. No standard data management procedures.


Here’s how bad data affects your sales teams:

  1. Data correction occupies a lot of time for sales teams.
  2. Sales representatives must manually enter a lot of data during sales calls, which is increasing call times.
  3. Sales teams must dig through multiple applications to find the data they need.
  4. Incorrect or outdated CRM data.

Business Intelligence

Check for these issues in your business intelligence:

  1. There are a lot of different data sets and spreadsheets in the department.
  2. Data mismatches cause people to distrust the data.
  3. Too many data sources.
  4. No data cleansing tools.
  5. The same information is stored in multiple formats.
  6. Variable definitions are inconsistent.

Human Resources

Look at how your human resources department is running:

  1. Correcting data is a regular part of the work process.
  2. Your human resources department has a high turnover rate.
  3. The hiring process is too long and often hires people that are a poor fit.

Supply Chain

Evaluate the efficiency and effectiveness of your supply chain:

  1. Correcting data is a standard part of your procurement process.
  2. Orders are sent to the wrong suppliers because supplier lists are unmanageable.
  3. Data is not consistent across all systems.
  4. Inability to get spending analytics because there is no centralized material information.
  5. Supplies are purchased, but never used.

Research and Development

How much work is research and development doing:

  1. Product improvement efforts are duplicated or redundant because product data is inaccurate or outdated.
  2. Customers don’t know about product improvements and changes.
  3. Research and development teams have inconsistent product specifications and standards, or no specifications and standards at all.
  4. Development and deployment cycles are excessively long.
  5. Marketing is out of sync with research and development efforts.


Find out what sort of errors and issues finance is having:

  1. Billing information inaccuracies cause payment delays and strain customer service resources.
  2. Customers get billed for cancelled services because CRM and financial systems have inconsistent data.
  3. Account profiles have incorrect payment and contact information.
  4. Customers receive products or services they didn’t pay for. Other customers don’t receive the goods they did pay for.

Information Tech (IT)

IT is responsible for data continuity:

  1. It’s impossible to keep data consistent across all systems.


Obviously, none of these issues alone will sink your ship.

However, each issue chips away at your efficiency. Enough of these problems can add up to a lot of dollars lost.

Of course, the snowballing effect also works in reverse. Improving your data will actually solve quite a few of these problems in one fell swoop.

So, how do you improve your data for big gains on your bottom line?

Verify Data at the Point of Entry

This is a big one because it largely involves your customer data. However, you can verify internal data at the point of entry as well.

People often forget to fill in all form fields or fail to enter complete data. This is the biggest issue with end-user data entry.

There’s nothing malicious about this. But it has widespread effects that can take a lot out of your business.

This is especially true for data that customers enter in form fields. Customers are notoriously bad at entering data. And customer data is one of the most important kinds.

Verifying data at the point of entry is the best way to protect your databases and save a lot of time on the back end.

The most efficient way to do this is to use an API to validate information in real time.

For the form fields that customers typically fill out, you can get a prebuilt API from a data verification company.

Then just plug and play the API on your customer facing form fields.

For internal data validation, you can use the same API in some cases.

For things like human resources, where you’re collecting mostly standard personal information, the same API that you use for customer form fields will work.

However, you may need to do a little bit of development to create a custom API for things like product specifications and other proprietary information validation.

Either way, checking the data as it comes into your databases will save you a lot of time and money in the long run.

The system will stop and prompt the user to correct errors or fill in incomplete fields. The data validation happens in the background and is entirely transparent to the user.

This will keep the bad data bugs out of your system.

Cleanse Your Databases

If you notice symptoms of bad data, you’ve got some bad information running around in your system already.

Validating incoming data will help. But you still need to remove the bad data that’s already made its way in.

In this case, you’ll need to do some data cleansing.

Cleansing the data itself is fairly straight forward. Simply compile and aggregate your data into a usable file—.txt, .csv, and Excel files are best—and hand it off to a data cleansing service for processing.

This can often be done entirely online.

Where things get tricky depends on how you use your data.

In some cases, invalid entries can be simply tossed out. A list of email addresses that you’ve collected using a lead magnet or trip wire? Just get rid of any invalid email addresses.

But say you’re a hospital that’s cleansing patient data. You obviously can’t just delete any patient entry with incorrect information.

In cases like this, you’ll most likely need to separate out the entries with errors. Then you’ll manually contact patients to get correct information.

The process of correcting the data can be arduous. But data cleansing makes it very easy to identify the bad data and create a hit list of entries that need to be corrected. So it still saves a lot of time.

Between validating your data at the point of entry and cleansing your existing lists, you should be in pretty good shape, datawise.

There’s just one last piece of the clean data puzzle.

Setup Data Management Procedures

Data management is all about how you treat the data in your databases.

Poorly organized data can behave a lot like bad data, even if all the information is correct.

This is especially true if you use a lot of bulk data processing or artificial intelligence for big data analytics.

The key is to create standardization for the important aspects of your databases.

Here are the big things to address in your data management policies:

  1. Data formatting. Uniform formatting ensures that both people and computers can find the right data when they need it.
  2. Adding data fields. This goes along with formatting. Always add new variables methodically.
  3. Automate processes. Automating data management processes reduces the margin for error. There’s a chance that errors will be introduced whenever people manually transfer data. So it’s best to let computers do the copy and pasting whenever you can.
  4. Continually cleanse and filter your data. In most businesses, it’s impossible to always handle data in a way that disallows the introduction of errors. So you’ll need a good system for consistently quality controlling your information.
  5. Backup your data. This one goes without saying. Always backup your data. One is the most dangerous number in business and in life.

All this will help keep your data healthy. But keep an eye on things once you’ve got all the bad data bugs worked out.

Bad data gets introduced into your databases as you expand your operations and adopt new software and systems. It’s almost inevitable.

So, periodically audit your organization. Check your various departments. Find out if your data has been significantly compromised.

If you find bad data issues, it may be time to do a little data spring cleaning and refresh your quality control measures.

What’s the most common problem you experience with your data?

Leave a comment and let us know what it is and how you solve it!



3 Government Databases You Must Know About to Stay Safe

3 Government Databases You Must Know About to Stay Safe

For over a decade, the United States has been on high alert to terrorist attacks, just as other parts of the world have too. There are so many world conflicts taking place that a country must take the proper precautions to protect its citizens.

In an effort to keep our country safe, the United States Government has put together several government databases listing known criminals, terrorists, products, and companies that may not do import and export business with the U.S. This also means that individuals or entities in the U.S. are forbidden to do business with any person or business that have been blocked, denied, and debarred and appear in these databases.

Where Do I Find the Government Databases?

There are websites that one can access easily to view the government databases. There are over 80 known lists to cross check whether a person or entity has been banned from doing business with the United States. When a U.S. company is conducting business with a foreign entity, it is the company’s responsibility to do their due diligence to ensure the safety of its country.

The U.S. Department of Homeland Security’s site will provide several links to lists providing information on blocked and denied entities, as well as a debarred persons lists.

The agencies the site refers to are the Bureau of Industry and Security which will provide a Denied Persons List and Entity List

The Office of Foreign Assets Control which provides a Special Designated Nationals and Blocked Person List

The Office of Defense Trade Controls, which gives a Debarred Parties List

The U.S. Government Printing Office, which will provide the Federal Register. 

  • Each of these agencies is responsible for keeping its lists up-to-date. The purpose of the lists is to provide exporters with companies, entities, and persons sanctioned by the U.S. Government and are forbidden to export goods from the U.S. It is the job of the exporter to be sure they are conducting transactions using the proper procedures and that exports are authorized properly.
  • The U.S. Department of State is another site that can provide useful information to ensure the safety of the country. The site provides links to the following lists: The Foreign Terrorist Organizations List (FTO), which provides a list of names of individuals or organizations that are or have been known to conduct business with terrorist groups or themselves are part of a terrorist group. Once these targets are listed, they will not be allowed to travel in the U.S. and any financial accounts with U.S. institutions are frozen.The State Sponsors of Terrorism is a list of countries that have been directly involved with support of terrorist groups. Currently there are four countries on this list, which are Cuba, Iran, Syria, and Sudan. Executive Order 13224, which was signed by President Bush in 2001, designates that the U.S. Government can impede terrorist funding. They have the power to stop financial support networks and block assets of foreigners or business that have given support to terrorists or pose a threat of doing so. The last link this site provides is to the Terrorism Designations Press Releases page, which lists the announcements from the Office of the Spokesperson as new terrorists or organizations are identified.
  • The U.S. Food and Drug Administration (FDA) provides a list of companies and individuals that are debarred under sections 306(a), (b)(1) and (b)(2) of the Federal Food, Drug, and Cosmetic Act, which is published in the Federal Register. Not only does this site provide the names of the offenders, but it gives the effective date they were added to the list, how long they are barred, and in most cases, a document about the case is provided from the Federal Register.

All of the sites referenced not only have links to the various lists, but also provide detailed information on how the lists are determined and used. Ultimately, it is the responsibility of individuals and businesses to use reasonable care and due diligence when conducting business affairs with foreign countries.


Using Online Databases to Find Doctors

Online Databases to Find Doctors
If you’ve recently moved to a new region, or you just want to switch doctors, you can start using online databases to find doctors. Internet databases make the process of finding the right doctor easier. Many such databases now exist, and many are free to use.

Reliable doctor databases should tell you mostly everything you need to know in order to make an informed decision, including a given doctor’s;

  • specialty
  • educational background, training and place of residency
  • years of experience
  • insurance coverage

 Where do I start my search for a new doctor?

You might start your search for a doctor by looking at a basic database which supplies the items listed above, such as the one maintained by the American Medical Association.

You could also try the WebMD Physician Directory. The information WebMD’s directory provides is updated twice every year, and includes directions to the offices of each physician it lists. If you’re on Medicare, you’ll find a list of Medicare-enrolled doctors through the government-run website called Physician Compare. On this site you can search for doctors according to a number of different criteria. You may search, for instance, for a doctor by gender, or for doctors who speak languages in addition to English.

Once you have put together a short list of doctors who are available to you, you’ll probably want to delve deeper and examine quality of care issues for each name on your list. Vitals.com is one website that can help you with this task. Its review system asks volunteer reviewers to rate doctors on intriguing criteria, such as: How accurate are a doctor’s diagnoses? How well does she follow up? How easy is it to make an appointment with him? How’s her bedside manner? The descriptions of doctor experiences which patients provide are often very detailed here.

Online Databases to Find Doctors

The Rate MDs website makes for compelling browsing as well. For one thing, it includes regional “top ten lists”: for example, the top ten best dentists – as reviewed by patients – near Wichita, Kansas. And its store of patient reviews is enormous, among the most extensive of any doctor database. Users can even add their own photos of doctors to the site.

It’s also important to search the Internet for malpractice databases – that is, databases that compile lists of malpractice complaints formally lodged against doctors. For instance, Health Grades is a public database informing site visitors of doctors’ histories of malpractice claims, including court rulings and settlement facts and figures.

And the federal government runs a similar database called the National Practitioner Data Bank. However, the NPDB, which comprehensively tracks malpractice actions taken against doctors, has been the subject of controversy.

In fact, the Health Resources and Services Administration (HRSA), kept this database shut down for a time, a move supported by the American Medical Association. And even when the NPDB went online again in late 2011, it did not provide the names of doctors with histories of malpractice lawsuits. Rather, it simply referred to those doctors by identification numbers. Many public health advocates and journalists, however, have fought for full disclosure on this website regarding those accused of malpractice. Whichever way this battle ultimately gets resolved, you should look at this database or similar databases to see what information you can glean.

A few words of warning are in order. Beware of medical databases which aggressively try to sell you medical products or insurance plans, and databases which doctors must pay to get listed on. And remember that patient reviews, like all reviews, are highly subjective. Different people bring to their doctors different expectations. Further, in most cases there’s no way to confirm that an online reviewer who claims to have been a patient of a particular doctor actually was a patient of that doctor.

In short, using databases to find doctors is a great start. Try reading at least several reviews, as together they should provide an overall impression of a certain doctor. On the other hand, don’t place too much trust in any single review.


Franchise Tax Board Asset Location

Are you trying to find out who recommends SearchBug.com?   It’s easy to say we’re trusted by financial institutions, hospitals, collection agencies, Marketing firms, and Fortune 500 Companies and have references and public reviews, but that only goes so far.

Who does the Franchise Tax Board turn to when they need Asset Location information? Well, according to their Collection Procedures Manual, it’s Searchbug.

The image below is taken directly out of the California Franchise Tax Board Collection Procedures Manual under the Debtor Asset Location Section (page 5).

It shows that Searchbug.com is actually the only Internet Referenced Third Party Contact Resource listed. If you want to download the entire 68 page manual, click the image.

Third Party Contact as listed in the Franchise Tax Board Collection Procedures Manual SearchBug.com

When State of California Collection Agents need to do some online investigations, their Procedures Manual references SearchBug.com as their third party contact.  We hope this shows that we’re a little different than other online data providers.  If we can be trusted by the state of California, we hope you can trust that we’ll do our best to help you too.

We answer our phones and emails in a timely manner; and we honestly believe you should be able to trust and communicate with the people you choose to do business with. Give us a try, we hope to help you soon.