Sometimes You Just Have to Purge (Your Data)

Let’s talk about data. There are big data, granular data, baseline data, quantitative data, sales data, product data … you get the picture. But the only data that really matter are sets relevant to you and your business. That can include a huge amount of data from a variety of applications that may or may not currently have a useful purpose.

When I work with companies on an initial system implementation — a new system, not an upgrade — one of the first things I like to do is look at their current data. Whether they’re in a system or a spreadsheet, it’s important to understand what types of data the company currently has, as well as which types of data they need to fill in the blanks in their new application. There can be up to three main types of data most companies work with in their applications:

Transactional Data 

Transactional data contain operational elements that support and help automate a business, including sales orders, point of sales data, purchase orders, inventory, manufacturing, billing, accounts receivable and payable. Typically, transactional data, such as purchase orders, sales orders, inventory quantities, invoices, etc. are all relevant for a specific period of time and need to be retained for accounting and audit purposes.

Master Data

This refers to organizational information that may be used across one or more systems and generally drives analytics across the business. Typically, master data are constant, static data utilized by multiple systems to define products, customers, vendors, suppliers, employees, locations and their attributes.

Analytical Data

This refers to the metrics, measurements and numerical values that deliver business intelligence and assist with organizational decision making. Analytical data can be pulled from multiple systems into decision support tools, such as a data warehouse or other reporting applications.

Not all applications contain transactional, master and analytical data, but it’s the combination of all three types that allow businesses to create reports and make and execute decisions. So how long do you need to retain all of that data? Unless you’re a brand-new company starting with a completely clean slate, you’ve probably already accumulated a lot of data in one or more of your systems that may no longer be relevant to your business. Some applications allow data to be hidden or only display current information, such as current and future product seasons, as well as a defined number of previous seasons where styles or inventory data may still be required. However, those data still reside in the database and take up digital space and reduce productivity. The amount of time you retain data can vary by application and type of data.

What do you do with the superfluous data, and why would you want to archive or purge them? Well, the more data you have, the slower the system may become. It takes a lot longer to churn through years of obsolete data just to find the information you need. Your databases may become unmanageable with outdated transactions, products, sales leads, etc. You may want to upgrade your application but don’t need to import all of the data from the older version. Some cloud-based applications may not have this issue, but not all applications are cloud-based, and your business will start to feel the crunch at some point.

Another key time to purge your data is when you divest one or more business divisions or brands. If you sell off part of your business, you may no longer have the legal right to the data and may be required to purge them. Deleting data from a screen doesn’t necessarily delete data from an application.

The benefits of archiving or purging data include increased system performance, manageable databases, simplified application upgrades and reduced IT costs with fewer hardware upgrades. Before you purge, though, consider the accounting and legal requirements for data retention, retaining data integrity and the service level or contractual agreements with customers, vendors or factories.

To archive or purge safely, consult your IT team to find out if you have the internal skill set to manage a data archive or purge. You should also consult your application’s support team to understand the best archive/purge methodology for the application, because each application may have different requirements. To borrow a phrase from carpentry, “Measure twice, cut once.” Enlist the support of IT professionals to manage the process, or you may run the risk of making an unrecoverable mistake.

It’s a good idea to review and purge certain data annually if there are no significant changes to your business. Publicly held companies have rules for retaining certain types of transactional data, such as invoices, purchase orders, sales orders, returned merchandise, etc. Other types of transactional data may be purged more frequently, such as pick tickets, shipment tracking numbers, etc. When purging data from applications, make sure to understand what other systems may also contain the same data. For example, product data may exist in product lifecycle management (PLM), enterprise resource planning (ERP), B2B and B2C websites, warehouse management systems (WMS), data warehouses or retail point of sale (POS) systems, plus forecasting and planning tools and more. The same data might not necessarily need to be purged from all applications, so make sure to work with your core business teams to understand their specific data needs and requirements.

Jill Mazur is an independent business process and technology consultant based in Los Angeles, California. She can be reached at

Sign Up for Newswire

    [ctct ctct-765 type:hidden 'Fashion Mannuscript Newswire::#156']