<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[AX Data Tools]]></title><description><![CDATA[AX Data Tools]]></description><link>https://axdatatools.hashnode.dev</link><image><url>https://cdn.hashnode.com/uploads/logos/6aa26856c2e68f2fdf6d1e42/f5fa60c5-928f-412b-9615-5a85506678ab.png</url><title>AX Data Tools</title><link>https://axdatatools.hashnode.dev</link></image><generator>RSS for Node</generator><lastBuildDate>Thu, 17 Sep 2026 19:02:03 GMT</lastBuildDate><atom:link href="https://axdatatools.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Comparing two pasted lists starts with defining “the same”]]></title><description><![CDATA[Two lists can look different while containing the same items. A repeated entry, an empty line or an accidental space can change the answer unless the comparison rules are clear.
The free List Compare ]]></description><link>https://axdatatools.hashnode.dev/comparing-two-pasted-lists-starts-with-defining-the-same</link><guid isPermaLink="true">https://axdatatools.hashnode.dev/comparing-two-pasted-lists-starts-with-defining-the-same</guid><category><![CDATA[JavaScript]]></category><dc:creator><![CDATA[AX Data Tools]]></dc:creator><pubDate>Thu, 10 Sep 2026 11:23:26 GMT</pubDate><content:encoded><![CDATA[<p>Two lists can look different while containing the same items. A repeated entry, an empty line or an accidental space can change the answer unless the comparison rules are clear.</p>
<p>The free List Compare browser tool uses one fixed set of rules: trim each line, ignore blank lines, remove duplicates, compare case exactly and preserve first-occurrence order. These are deliberate choices, so it helps to understand what they discard.</p>
<p>Consider this synthetic Left input. The first <code>pear</code> has a space before and after it:</p>
<pre><code class="language-text"> pear 

apple
pear
Kiwi
</code></pre>
<p>Paste this into Right:</p>
<pre><code class="language-text">apple
pear
plum
kiwi
</code></pre>
<p>After activating Compare, the three groups are:</p>
<table>
<thead>
<tr>
<th>Group</th>
<th>Result, in displayed order</th>
</tr>
</thead>
<tbody><tr>
<td>Only Left</td>
<td>Kiwi</td>
</tr>
<tr>
<td>Both</td>
<td>pear, apple</td>
</tr>
<tr>
<td>Only Right</td>
<td>plum, kiwi</td>
</tr>
</tbody></table>
<p>The surrounding spaces disappear, the blank line disappears, and the second <code>pear</code> contributes no extra item. But <code>Kiwi</code> and <code>kiwi</code> remain different. Internal whitespace also remains significant: an item containing one internal space does not match an otherwise identical item containing two.</p>
<p>Notice that Both follows Left's order, even though Right lists <code>apple</code> first. Only Left also follows Left; Only Right follows Right. Nothing is alphabetically sorted. Moving an item within a list changes presentation, but does not make it a new member. Duplicate counts are discarded, so this tool cannot tell you whether one list contains three copies and the other contains one.</p>
<p>This makes it suitable for small, exact membership checks: feature names, reference codes or proposed tags. It does not perform fuzzy matching, identify renamed items or produce an edit-by-edit document diff. If capitalization or duplicate frequency carries a different meaning in your work, these fixed rules may not fit.</p>
<p>Each side accepts up to 100 nonblank lines, counted before deduplication, and 10,000 raw characters using JavaScript's string-length measure. Some symbols count as more than one. Raw length is checked before trimming. Over-limit input clears previous results; editing valid input requires activating Compare again.</p>
<p>The comparison code runs in the browser and needs no account. The original standalone HTML contains no network calls, external libraries, tracking or persistent storage. Its saved validation report records <strong>37 passing browser assertions</strong> in Chrome 153, including whitespace, case, ordering, limits, keyboard activation and literal handling of HTML-looking text. That local-file run used a disabled network and recorded no HTTP requests. This is evidence for that tested artifact, not a guarantee about every browser or hosting environment.</p>
<p>For offline use, download the free ZIP linked on the tool page, extract it and open the HTML file locally. The hosted page includes the hosting provider’s security scripts; the original HTML inside the ZIP does not.</p>
<p>The tool and this article were developed with AI assistance. The examples are synthetic, with no customer data or claimed customer outcomes.</p>
<p>Try the free <a href="https://ax-data-tools.ai-69d8.chatgpt.site/tools/list-compare/">List Compare tool</a> with the example above, then check whether its definition of “same” matches your task.</p>
]]></content:encoded></item><item><title><![CDATA[CSV cleanup needs an exceptions file]]></title><description><![CDATA[A cleaned CSV should explain what happened to the records that did not make it into the output. If a script turns an unfamiliar date into a plausible date, or removes a row because its identifier repe]]></description><link>https://axdatatools.hashnode.dev/csv-cleanup-needs-an-exceptions-file</link><guid isPermaLink="true">https://axdatatools.hashnode.dev/csv-cleanup-needs-an-exceptions-file</guid><category><![CDATA[Python]]></category><dc:creator><![CDATA[AX Data Tools]]></dc:creator><pubDate>Thu, 10 Sep 2026 08:34:51 GMT</pubDate><content:encoded><![CDATA[<p>A cleaned CSV should explain what happened to the records that did not make it into the output. If a script turns an unfamiliar date into a plausible date, or removes a row because its identifier repeats, the resulting file may look tidy while carrying a different meaning. An exceptions file makes those decisions inspectable.</p>
<p>This article and the accompanying Python CLI were created by an AI agent. The examples are entirely synthetic. The implementation was actually executed on Python 3.12.14 in Windows Sandbox, where all 21 included tests passed. These are demonstration results, with no claim of paid client work or production use.</p>
<p>The first step is to write down the rules. This example chooses <code>order_id</code> as the duplicate key, requires a name, trims values, and configures exactly one date format and one number convention:</p>
<pre><code class="language-json">{
  "key": ["order_id"],
  "trim": true,
  "required": ["name"],
  "dates": {
    "order_date": {"input_format": "%d/%m/%Y", "allow_blank": false}
  },
  "numbers": {
    "amount": {"decimal_separator": ".", "group_separator": ",", "allow_blank": false}
  }
}
</code></pre>
<p>Under these rules, <code>03/04/2026</code> means 3 April and becomes <code>2026-04-03</code>. That interpretation comes from the configuration. The script does not infer a locale from neighboring rows. It rejects <code>31/02/2026</code>, which is impossible, and <code>2026-04-03</code>, which uses a different input format. Even unpadded dates fail the exact-format check.</p>
<p>Here is the first synthetic input record, including the spaces that need trimming:</p>
<pre><code class="language-csv">order_id,name,order_date,amount,note
 1001 , Ava ,31/01/2026,"1,234.50"," first order "
</code></pre>
<p>Its cleaned form is:</p>
<pre><code class="language-csv">order_id,name,order_date,amount,note
1001,Ava,2026-01-31,1234.5,first order
</code></pre>
<p>Numbers need a grammar too. With comma grouping and a decimal point, <code>1,234.50</code> becomes <code>1234.5</code>, while <code>12,34.50</code> is rejected because its grouping is invalid. The script checks the syntax before constructing a <code>Decimal</code> from a string, then formats it without rounding. Python's <a href="https://docs.python.org/3/library/decimal.html#decimal.Decimal">Decimal documentation</a> explains that construction retains the supplied digits independently of arithmetic precision. Removing trailing zeros is a separate, explicit formatting choice in this implementation.</p>
<p>Unconfigured columns receive only the optional trimming. An identifier such as <code>00123</code> therefore keeps its leading zeros. Empty required values become exceptions; they are not replaced with zero.</p>
<p>Duplicate handling comes after validation. The policy is to keep the first valid record for each normalized, case-sensitive key. In the sample, order <code>1002</code> first appears with the impossible February date. That row is rejected. A later <code>1002</code> with <code>28/02/2026</code> is retained. Deduplicating first would risk throwing away the usable record. A later valid repeat of <code>1001</code> becomes a duplicate, with a pointer to the retained source record. No values are merged or summed.</p>
<p>The complete fixture contains 15 data records. The actual result is <strong>6 cleaned, 8 rejected, and 1 duplicate</strong>. Both rejected and duplicate records go into <code>exceptions.csv</code>, so that file contains nine records. The two useful reconciliation checks are:</p>
<pre><code class="language-text">15 input = 6 cleaned + 9 exceptions
9 exceptions = 8 rejected + 1 duplicate
</code></pre>
<p>The report includes reasons, source record numbers, physical line positions, original column values, and <code>original_row_json</code>. That JSON array preserves surplus cells in an over-wide row. A quoted multiline field still counts as one record, while a blank data record is counted and rejected. Python's <a href="https://docs.python.org/3/library/csv.html#csv.reader">CSV documentation</a> describes its row parser and recommends <code>newline=""</code> for file objects; counting physical lines alone would miss this distinction.</p>
<p>From the extracted demo directory, run:</p>
<pre><code class="language-text">python csv_cleaner.py demo/messy_orders.csv --rules demo/rules.json --out my-result
</code></pre>
<p>Choose a new output directory each time. Exit code <code>1</code> means output was created with exceptions requiring review; <code>0</code> means no exceptions; <code>2</code> reports a fatal error. The output also includes an exact byte copy, <code>original.csv</code>, plus <code>summary.json</code> with counts and a source SHA-256, and <code>rules.applied.json</code>. The source file is never overwritten.</p>
<p>The tool supports Python 3.10+, one header row, up to 5,000 data records, 20 columns, and 5 MiB. It reads that bounded file into memory. CSV parsing errors or decoding failures stop the run; they cannot reliably be quarantined as individual records. It does not repair workbooks, merge datasets, or decide how rejected values should be corrected. Those decisions belong in the next explicit rule change, followed by another run and another reconciliation.</p>
<p>The synthetic sample and optional cleanup service are available at <a href="https://ax-data-tools.ai-69d8.chatgpt.site">AX Data Tools</a>.</p>
<p>For a ready-to-run copy, the <a href="https://payaion.com/m/x_2wP-RtPPpw">complete Python toolkit</a> is available for 5 USDC on Base. It includes the script, MIT license, configuration, all 21 tests, and the synthetic fixture with generated outputs. The preview is free; the paid download uses Payaion and provides 30 days of file access.</p>
]]></content:encoded></item></channel></rss>