Data Storage Optimization System

Client: One of the world’s leading data warehousing and cloud technologies solutions providers

Task: The customer needed to address the performance issue with the existing long-term data storage management product. One of the tasks was to do a triage and develop a mechanism for selecting and archiving the rarely used data. The task was not limited to transfer or copy information but to also extend the application capabilities by adding the tools to manage a full-scale data migration.

Solution: The product developed by our team makes it possible to specify the locations to be processed, the processing conditions and the destination archive. The system generates directory status reports before and after processing, with an option to simulate processing for increased data security. The product supports connecting several machines (agents) to the same server.

A part of the project was to provide transparent access to the program modules that remain in place of the original file. The mechanism used were adjusted according to specifics of the target application. For instance, symbolic references were used in Unix, while in order to support similar mechanisms in Windows and NetApp our own solutions had to be implemented.

The product includes:

  • Web application for system administration (setting the search criteria, work area, archives, report visualization etc.)
  • Service for scanning file systems and processing the files that meet certain criteria
  • Windows Explorer Shell Extension to process files directly and check their status
  • Filters for Windows and FPolicy Server for NetApp for transparent access to migrated files

Technologies: PHP, Apache web server, C, openssl, libxml, CrashRpt, PostgreSQL, MSWindows, RedHat