Skip to content
Astera Technology

Why does an imported name contain a replacement character?

Reject malformed source bytes before a permissive decoder creates apparently usable text.

Download complete case notes and answer keys ↓

Read the source and its declared rule

This fixture requires valid UTF-8. Its bytes C3 28 contain an incomplete two-byte lead followed by an invalid continuation. A permissive decoder replaces the invalid part; the original import rule instead holds that source with its bytes intact. Empty encoded text is allowed here.

Filled synthetic input
RecordInput JSON
Base input{"bytes":[195,40]}
Download editable CSV ↓

Compare the actual paths

A displayed replacement does not recover the missing intended character. Corrected output rejects the record and retains [195,40]; do not repair a person’s spelling by guessing from that display.

Misleading and corrected outputs
PathOutput JSON
Misleading path{"text":"\uFFFD(","accepted":true}
Corrected path{"accepted":false,"error":"invalid-utf8","bytes":[195,40]}
Download editable CSV ↓
Local Node.js program: 22.23.3 or 24.14.1
import {DatabaseSync} from 'node:sqlite';
import {TextDecoder} from 'node:util';
import {inflateSync} from 'node:zlib';
import {posix} from 'node:path';
import {createHash} from 'node:crypto';
const rows=(db,sql)=>db.prepare(sql).all().map(r=>({...r}));
const failed=(action)=>{try{action();return null;}catch(e){return {code:e.errcode??e.code??e.name};}};

import assert from 'node:assert/strict';
const run=function run(i,corrected=true){
    try{return {text:new TextDecoder('utf-8',{fatal:corrected}).decode(Uint8Array.from(i.bytes)),accepted:true};}
    catch{return {accepted:false,error:'invalid-utf8',bytes:i.bytes};}
  };
const cases=[{"label":"misleading base","input":{"bytes":[195,40]},"expected":{"text":"\uFFFD(","accepted":true},"corrected":false},{"label":"corrected base","input":{"bytes":[195,40]},"expected":{"accepted":false,"error":"invalid-utf8","bytes":[195,40]},"corrected":true},{"label":"valid accented character","input":{"bytes":[195,169]},"expected":{"text":"é","accepted":true},"corrected":true},{"label":"empty encoded text","input":{"bytes":[]},"expected":{"text":"","accepted":true},"corrected":true}];
for(const test of cases){
  const actual=run(test.input,test.corrected);
  assert.deepEqual(actual,test.expected,test.label);
  console.log(test.label,actual);
}
Download runnable example ↓

Check boundaries before the next load

The valid accented byte pair decodes to é. Empty bytes decode to an empty string without a decoding error.

Editable boundary records and expected outputs
CaseInput JSONExpected output JSON
Valid accented bytes{"bytes":[195,169]}{"text":"é","accepted":true}
Empty byte input{"bytes":[]}{"text":"","accepted":true}
Download editable CSV ↓

Can the bad displayed text establish the original name?

Show the worked answer

No. It contains a substituted character, not the lost source meaning. Hold this input; the valid é boundary is accepted.

Related resources