Editing Portal:Databases (section)

=== CSV, spreadsheets, logs, and semi-structured data ===

Perl’s role in data development has never been limited to formal databases. A great deal of real data lives in CSV files, spreadsheets, log files, exported reports, mailboxes, fixed-width text, JSON, XML, YAML, and poorly documented vendor formats. Perl’s regular expressions and CPAN ecosystem made it especially good at converting these formats into database-ready records.

Common Perl data workflows include:

* importing CSV files into MySQL, MariaDB, PostgreSQL, or SQLite;
* cleaning inconsistent names, dates, addresses, codes, and identifiers;
* parsing logs and producing summary tables;
* converting legacy flat files into normalized relational schemas;
* extracting data from HTML, XML, JSON, or email;
* comparing database exports between systems;
* generating reports from SQL queries;
* building searchable indexes from messy text;
* automating backups, dumps, restores, and migrations.

This “between systems” role is one of Perl’s most durable strengths. Perl is often not the database and not the final application. It is the layer that understands both sides well enough to move the data safely.