Skip to content
Astera Technology

Why do an AB export’s bytes become different characters?

Use the declared UTF-16 byte order and hold a contradictory or incomplete representation.

Download complete case notes and answer keys ↓

Read the source and its declared rule

The supplied file contract is UTF-16 LE with complete two-byte units. Bytes 41 00 42 00 mean AB under that contract. Decoding the same bytes as UTF-16 BE yields different characters. The corrected path checks this file’s declaration and even byte length.

Filled synthetic input
RecordInput JSON
Base input{"encoding":"utf-16le","bytes":[65,0,66,0]}
Download editable CSV ↓

Compare the actual paths

The correction produces AB. Rejecting a contradictory BE declaration is a rule for this supplied file, not a claim that BE is an invalid encoding generally.

Misleading and corrected outputs
PathOutput JSON
Misleading path{"accepted":true,"text":"䄀䈀"}
Corrected path{"accepted":true,"text":"AB"}
Download editable CSV ↓
Local Node.js program: 22.23.3 or 24.14.1
import {DatabaseSync} from 'node:sqlite';
import {TextDecoder} from 'node:util';
import {inflateSync} from 'node:zlib';
import {posix} from 'node:path';
import {createHash} from 'node:crypto';
const rows=(db,sql)=>db.prepare(sql).all().map(r=>({...r}));
const failed=(action)=>{try{action();return null;}catch(e){return {code:e.errcode??e.code??e.name};}};

import assert from 'node:assert/strict';
const run=function run(i,corrected=true){
    if(corrected&&(i.encoding!=='utf-16le'||i.bytes.length%2!==0))return {accepted:false,error:'encoding-contract'};
    try{return {accepted:true,text:new TextDecoder(corrected?i.encoding:'utf-16be',{fatal:true}).decode(Uint8Array.from(i.bytes))};}
    catch{return {accepted:false,error:'invalid-encoded-text'};}
  };
const cases=[{"label":"misleading base","input":{"encoding":"utf-16le","bytes":[65,0,66,0]},"expected":{"accepted":true,"text":"䄀䈀"},"corrected":false},{"label":"corrected base","input":{"encoding":"utf-16le","bytes":[65,0,66,0]},"expected":{"accepted":true,"text":"AB"},"corrected":true},{"label":"contradictory declaration","input":{"encoding":"utf-16be","bytes":[65,0]},"expected":{"accepted":false,"error":"encoding-contract"},"corrected":true},{"label":"odd trailing byte","input":{"encoding":"utf-16le","bytes":[65]},"expected":{"accepted":false,"error":"encoding-contract"},"corrected":true}];
for(const test of cases){
  const actual=run(test.input,test.corrected);
  assert.deepEqual(actual,test.expected,test.label);
  console.log(test.label,actual);
}
Download runnable example ↓

Check boundaries before the next load

A contradictory declaration and one trailing byte both fail before a record is accepted.

Editable boundary records and expected outputs
CaseInput JSONExpected output JSON
Conflicting byte-order declaration{"encoding":"utf-16be","bytes":[65,0]}{"accepted":false,"error":"encoding-contract"}
Incomplete byte pair{"encoding":"utf-16le","bytes":[65]}{"accepted":false,"error":"encoding-contract"}
Download editable CSV ↓

Should a readable-looking wrong-order result be imported?

Show the worked answer

No. Readability is not the declared codec. This source requires LE and complete byte pairs, producing AB.

Related resources