WHAT ARE THEY?

Basically a data source (or store) is a collective term that can refer to any type of file that stores data. This not only includes traditional data sources like databases, but also other types of files that store data like text files, spreadsheets and other documents.

The traditional and the most widely used and supported data source is the database. A database contains highly structured data that can be retrieved and managed easily. Reporting software has to retrieve data from various such data sources to create a report. The aim of this article is to provide a simple introduction to the various types of data sources available today.

WHICH ARE THEY?

Here is a list that shows the most common sources of data. Bear in mind though that these sources of data may or may not be located on the same PC or even in the same office.

1. Databases: The most commonly used sources. This is what reporting software’s most usually read to fetch data. The methods and software protocols required to access data from these are standardized and therefore, all reporting software(s) support them. Databases also include structures like indexes, hash tables, security descriptors, metadata etc which increase data access speed and retrieval efficiency while at the same time even serve to protect the very same data from damage and/or unauthorized access.

   Practical Implications

   A wide variety of software and related literature exists which makes the process of integrating a database system with your reporting software almost seamless and without any hassles. Reporting systems themselves inherently support the most common types of databases. While certain database formats are proprietary and require the use of vendor recommended software to effectively manage them; many are available under the General Public License (GPL) and are free to use as well.

2. Text files: These include XML (eXtended Markup Language) files as well other structured files (like CSV – Comma Separated Values) that add minimal amount of structural integrity to a data file.

   Practical Implications

   These files were created to be compatible across all types of computer systems. Since data exists in a structured way in simple text file, these can be viewed and edited directly (in most cases) via a simple text editor. As these are widely supported, these can be used as an intermediate format, when transferring data from one database system to some other.

   Storing native data in this format has huge security risks. Retrieval and management of data from these files takes more time when compared to a proper database system. Their only true advantage is their extreme portability. Reporting software(s) natively support them.

3. Other semi structured files from 3^rd party software: Files like spreadsheets, simple embedded tables in documents, the metadata information contained in almost any file etc are some of the examples of this. Data from these files can be directly read by reporting software provided that the appropriate “support software” is present on the system.
4. Custom formats: In rare cases some users might have a unique requirement. For example: A production manager in a manufacturing unit might want the reporting software to directly communicate with the manufacturing machinery to get data from the on board chip of that very machine. The reporting software has to be custom built to support such a feature.

A SPECIAL MENTION: THE DATA WAREHOUSE

Not all reporting software(s) support all the known database systems and their formats. This is where a data warehouse comes handy.

Imagine a scenario where the accounts data of a company is stored in a custom file managed by proprietary management software. However, that very software supports exporting of the data to a popular format (like CSV or to a spreadsheet for example).

In this case however, the reporting software of that company will have to do the following:

1. Extract data from the file.
2. Convert it into a standard format which makes it speedier and more convenient to manage.

This can be a bit time consuming for the software itself if the data required is spread over multiple geographic locations and is stored in a different format for each location.

The goals of a data warehouse are as follows:

1. Extraction of all the data from all data sources.
2. Formatting the said data and converting it to a standard format.
3. Enforce security constraints and make all data more easily available.

The advantages of this approach are obvious:

1. Data Independence: Even if the data source at the original location is corrupted or destroyed, users can still access data from the warehouse.
2. A single communication protocol can be used to access all the data in the company.
3. Corrections and formatting to the data are automatically applied to the data from the source when adding it to a data warehouse.

Therefore the data warehouse is basically a central repository of data that reporting and or other software(s) can use to access all the data it needs without being concerned about the actual location and/or format of the original data. The data warehouse presents the data in standard format to a reporting system.

The fact that a data warehouse can be used to access all the structured data a company has leads to some other interesting uses:

1. Data Mining: The data from different sources can be correlated and logical inferences can be drawn. For example: Manufacturing unit A has suffered has suffered from X hours – total downtime leading to an average loss of Y production units. Therefore manufacturing units B, C and D can work for Z more hours each day to offset the loss.
2. Data warehouses can also be used to store historical changes to data. (For example: Head of the accounts department from date A to B was Mr. X and from B to C was Mr. Y.) Data mining can be applied to this historical data as well.
3. As an indirect advantage, since communication between the users and native data sources is reduced, this leads to reduced usage of communication resources at the source.

To sum it up, a lot goes under the hood even for a simple task like retrieval of data. The end users simply remain oblivious of all these facts. Even when performing these tasks, the retrieval system itself has to follow a lot of protocols to ensure security and correctness of the data retrieved. We will look into a little more detail about all this in the next article.

THE REPORT

A report in simple terms is a highly subjective document that is intended for a specific audience and is usually very aggregative in nature. The above definition, when considered piece by piece becomes easier to understand. Here are the important points about the definition:

* Highly subjective: The document focuses on a subject at hand and may be highly technical in nature. A production report for a manufacturing enterprise contains various aggregates and averages calculated over time – for production figures only. It won’t contain sales, purchase or any other information.

Documents like purchase reports, production reports, sales reports, dispatch reports, expense reports, attendance reports, tax reports, etc are present in almost any office. Some of them are made manually whereas some are made automatically by the use of various reporting software. A “reporting software” has just one simple job: minimize manual intervention and labor in the process of report creation and at the same time, maintain the consistency, formatting and accurateness of the generated report.

Reporting software simply pulls data and figures from various sources, formats it and then creates a report as required. The above process however is not always as easy as it sounds. This is because of the following reasons:

FIGURE 1: A POSSIBLE SITUATION: THE REPORTING SOFTWARE HAS TO COLLECT DATA FROM VARIOUS SOURCES THAT MAY NOT BE LOCATED ON THE SAME PC.

1. The large variety of data sources: We did say that a “Reporting software simply pulls data”, but the data sources may not be of the same type. A reporting software may pull the attendance record of employees from one database combine it with salary records from another database (which may differ in format from the earlier one) and then, format the data and create a report that looks visually appealing.
2. Geographical location of data sources: The attendance and salary data may not be located on the same PC. In that case, the reporting software has to fetch the same via some other communication channels.
3. Formatting of the report: The word “formatting” includes but is not limited to, line setting, font adjustment, paper size adjustments, ensuring the adherence to a predefined template, conversion of data, calculation of aggregate figures, generation and placement of graphs, charts and other visual data if required etc. Clearly, reporting software has to do a lot of work to do while creating a report.

Other duties of reporting software might also include automatic printing of the generated report, record keeping of generated reports, automatic backups etc.

THE NEED OF REPORTING SOFTWARE

Let us consider a real life example: A production manager for an enterprise that manufactures tin cans has to send the daily production figures to the managerial staff of the company. The reports are regularly emailed to the head office that is located in a different city. The report also contains other data like machine downtime, raw materials used, details of rejected batches etc.

The manufacturing unit has 4 manufacturing machines that produce separate types of tin cans. Each machine has an onboard computer that keeps a record of the number of tin cans produced and sends the said data to a central server in the production manager’s office. The other figures like downtime, defective products etc are manually entered by the production staff.

The manager manually formats and creates the report thrice a day. On an average, the preparation and checking of each report takes about 20 minutes. Therefore the manager has to devote an hour each day to this task. As can be seen, most of the data is available electronically already and so, there is scope for automation here.

This entire process can be automated by the use of reporting software. All the manager has to do is verify the contents once before emailing it. This step too can be automated once a considerable amount of confidence is gained through repeated use of the software. The result – increased productivity. The manager does not have to spend an entire hour doing repetitive tasks. He/she can better utilize their time elsewhere.

Reporting software will in the long run further reduce manual labor and human intervention, when the users become more experienced in its use. All of the above mentioned advantages and conveniences are what make investment in reporting software worthwhile.