Field Encoding Types

Lesson, slides, and applied problem sets.

Lesson

Module 3: Field Encoding Types

The Encoding Problem

You've parsed the MTI. You've decoded the bitmap and know exactly which fields are present: 2, 3, 4, 11, 12, 22, 35, 41, 42, 49. Now comes the hard part: actually reading those fields from the wire.

Here's the challenge: ISO8583 fields vary wildly in their data types and sizes.

Field  Name                    Type        Example Value
─────  ────                    ────        ─────────────
DE2    Primary Account Number  LLVAR n..19 "4532015112830366"
DE3    Processing Code         Fixed n 6   "000000"
DE4    Amount                  Fixed n 12  "000000010000"
DE11   STAN                    Fixed n 6   "123456"
DE35   Track 2 Data            LLVAR z..37 "4532015112830366D2512..."
DE41   Terminal ID             Fixed ans 8 "TERM0001"

Notice the patterns:

Some fields are fixed-length: DE3 is always exactly 6 characters
Some fields are variable-length: DE2 (PAN) can be 13-19 digits
Some are numeric only: DE4 contains only digits
Some allow any character: DE41 can have letters, numbers, symbols

The ISO8583 specification defines a precise encoding for each field. Get it wrong, and your parser either crashes or produces garbage.

Understanding Field Specifications

Before diving into encoding types, let's understand how fields are specified. The ISO8583 standard uses a notation like:

n 6        = numeric, fixed 6 characters
n..19      = numeric, variable up to 19 characters
an 12      = alphanumeric, fixed 12 characters
ans..40    = alphanumeric+special, variable up to 40 characters
b 8        = binary, fixed 8 bytes
b..999     = binary, variable up to 999 bytes

The prefix tells you the data type:

n = Numeric (digits 0-9 only)
a = Alpha (letters only)
an = Alphanumeric (letters and digits)
ans = Alphanumeric + Special (printable ASCII)
b = Binary (raw bytes)
z = Track 2 code set (digits, =, separator characters)

The suffix tells you the length:

6 = Exactly 6 characters (fixed)
..19 = Up to 19 characters (variable)
...999 = Up to 999 characters (3-digit length prefix)

Variable-length fields need a length prefix so the parser knows where they end. That's where LLVAR and LLLVAR come in.

Fixed-Length Fields

Fixed-length fields are straightforward: read exactly N bytes, interpret according to data type.

Numeric Fixed Fields (n)

DE3: Processing Code - n 6
Wire:  30 30 30 30 30 30  (ASCII)
Value: "000000"

DE4: Amount - n 12
Wire:  30 30 30 30 30 30 30 31 30 30 30 30  (ASCII)
Value: "000000010000" = $100.00 (with 2 decimal places)

For numeric fields in ASCII encoding, each digit takes 1 byte (0x30-0x39).

The padding trap: Numeric fields are typically right-justified with leading zeros. A $100.00 amount in a 12-character field becomes 000000010000, not 100000000000 or 10000.

Alphanumeric Fixed Fields (an, ans)

DE41: Terminal ID - ans 8
Wire:  54 45 52 4D 30 30 30 31  (ASCII)
Value: "TERM0001"

Alphanumeric fields are typically left-justified with trailing spaces:

DE42: Merchant ID - ans 15
Wire:  4D 45 52 43 48 30 30 31 20 20 20 20 20 20 20
Value: "MERCH001       " (7 characters + 8 spaces)

Binary Fixed Fields (b)

Binary fields contain raw bytes, not character-encoded data:

DE52: PIN Data - b 8
Wire:  A1 B2 C3 D4 E5 F6 A7 B8
Value: [0xA1, 0xB2, 0xC3, 0xD4, 0xE5, 0xF6, 0xA7, 0xB8]

No character interpretation - just 8 raw bytes.

The Go Implementation

// ReadFixedField reads exactly 'length' bytes and interprets as string
func ReadFixedField(data []byte, offset int, length int) (string, int, error) {
    if offset+length > len(data) {
        return "", offset, fmt.Errorf("not enough data: need %d bytes at offset %d, have %d",
            length, offset, len(data))
    }
    value := string(data[offset : offset+length])
    return value, offset + length, nil
}

// ReadFixedBinary reads exactly 'length' bytes as raw binary
func ReadFixedBinary(data []byte, offset int, length int) ([]byte, int, error) {
    if offset+length > len(data) {
        return nil, offset, fmt.Errorf("not enough data for binary field")
    }
    result := make([]byte, length)
    copy(result, data[offset:offset+length])
    return result, offset + length, nil
}

Variable-Length Fields: LLVAR and LLLVAR

Variable-length fields solve a fundamental problem: how does the parser know where DE2 (PAN) ends and DE3 begins when the PAN can be 13-19 digits?

The answer: prefix the value with its length.

LLVAR: 2-Digit Length Prefix

LLVAR fields have a 2-character length prefix, supporting values up to 99 characters.

DE2: Primary Account Number - LLVAR n..19

Wire (ASCII):  31 36 34 35 33 32 30 31 35 31 31 32 38 33 30 33 36 36
               ├──┤ └─────────────────────────────────────────────────┘
               "16"            "4532015112830366"
               ↑               ↑
               Length=16       Actual PAN (16 digits)

Decoded: PAN = "4532015112830366"

The first 2 bytes are "16" (ASCII 0x31, 0x36), telling the parser to read the next 16 bytes as the value.

LLLVAR: 3-Digit Length Prefix

LLLVAR fields support values up to 999 characters, commonly used for large or structured data:

DE63: Private Data - LLLVAR ans...999

Wire:  30 34 35 ...45 bytes of data...
       ├────┤ └─────────────────────┘
       "045"        45 bytes of value
       ↑
       Length=45

LLLLVAR: 4-Digit Length Prefix (Extension)

Some implementations support LLLLVAR for values up to 9999 characters:

DE127: Network-Specific Data - LLLLVAR ans...9999

Wire:  31 32 33 34 ...1234 bytes...
       ├───────┤
       "1234"

ASCII vs BCD Length Prefixes

Here's a critical implementation detail: the length prefix itself can be encoded in ASCII or BCD.

ASCII Length Prefix:

Length 16 = "16" = 0x31 0x36 (2 bytes)
Length 45 = "045" = 0x30 0x34 0x35 (3 bytes)

BCD Length Prefix:

Length 16 = 0x16 (1 byte, packed BCD)
Length 45 = 0x00 0x45 (2 bytes) or 0x045 depending on spec

Most modern implementations use ASCII length prefixes, but you'll encounter BCD in legacy systems and some network variants.

The Go Implementation

// ReadLLVAR reads a 2-digit length-prefixed variable field
func ReadLLVAR(data []byte, offset int) (string, int, error) {
    // Need at least 2 bytes for length
    if offset+2 > len(data) {
        return "", offset, fmt.Errorf("not enough data for LLVAR length prefix")
    }

    // Parse 2-character ASCII length
    lengthStr := string(data[offset : offset+2])
    length, err := strconv.Atoi(lengthStr)
    if err != nil {
        return "", offset, fmt.Errorf("invalid LLVAR length: %q", lengthStr)
    }

    // Validate length is reasonable
    if length < 0 || length > 99 {
        return "", offset, fmt.Errorf("LLVAR length out of range: %d", length)
    }

    // Read the value
    valueStart := offset + 2
    if valueStart+length > len(data) {
        return "", offset, fmt.Errorf("LLVAR value truncated: need %d bytes", length)
    }

    value := string(data[valueStart : valueStart+length])
    return value, valueStart + length, nil
}

// ReadLLLVAR reads a 3-digit length-prefixed variable field
func ReadLLLVAR(data []byte, offset int) (string, int, error) {
    if offset+3 > len(data) {
        return "", offset, fmt.Errorf("not enough data for LLLVAR length prefix")
    }

    lengthStr := string(data[offset : offset+3])
    length, err := strconv.Atoi(lengthStr)
    if err != nil {
        return "", offset, fmt.Errorf("invalid LLLVAR length: %q", lengthStr)
    }

    if length < 0 || length > 999 {
        return "", offset, fmt.Errorf("LLLVAR length out of range: %d", length)
    }

    valueStart := offset + 3
    if valueStart+length > len(data) {
        return "", offset, fmt.Errorf("LLLVAR value truncated: need %d bytes", length)
    }

    value := string(data[valueStart : valueStart+length])
    return value, valueStart + length, nil
}

Common Bugs with Variable-Length Fields

Bug 1: Including the length prefix in the length

// WRONG: Length includes prefix
value := data[offset : offset+length]  // This includes "16" in the output

// RIGHT: Skip the prefix
value := data[offset+2 : offset+2+length]  // Start after the 2-byte prefix

Bug 2: Single-digit lengths

What if the length is "5"? Is it encoded as:

"05" (2 characters with leading zero) ← Correct
"5 " (single digit with trailing space) ← Some broken implementations
" 5" (single digit with leading space) ← Other broken implementations

The standard says leading zeros: "05". But test against your network's actual behavior.

Bug 3: Maximum length validation

// PAN is LLVAR n..19 - max 19 characters
// But what if wire says length is 99?

if length > maxAllowed {
    return "", offset, fmt.Errorf("DE2 length %d exceeds maximum %d", length, maxAllowed)
}

Always validate against the field's maximum length. Don't trust the wire data.

BCD Encoding: Packing Digits

Binary-Coded Decimal (BCD) is an encoding that packs two decimal digits into a single byte. It was designed to save bandwidth when every byte counted.

The Basic Idea

In ASCII, the digit "5" is byte 0x35. The number "12345" takes 5 bytes.

In BCD, each digit uses only 4 bits (a "nibble"):

Digit 0 = 0000
Digit 1 = 0001
Digit 5 = 0101
Digit 9 = 1001

Pack two nibbles per byte: "12" = 0001 0010 = 0x12

The number "12345" in BCD:

Digits:   1    2    3    4    5
Nibbles: 0001 0010 0011 0100 0101
Bytes:     0x12     0x34    0x05
           ↑        ↑       ↑
           "12"     "34"    "5" (padded with 0 nibble)

Wait - we have an odd number of digits. What happens to the last one?

Odd-Length Padding

When you have an odd number of digits, you need padding. There are two conventions:

Left-padding (more common):

"12345" (5 digits) → pad to 6 → "012345"
BCD: 0x01 0x23 0x45
First nibble is 0, then 1, 2, 3, 4, 5

Right-padding:

"12345" (5 digits) → "123450"
BCD: 0x12 0x34 0x50
Last nibble is 0 (padding)

Left-padding is standard for numeric values. Right-padding is sometimes used for fields like PAN where the length is preserved.

Encoding BCD in Go

// EncodeBCD converts a numeric string to packed BCD bytes
// Uses left-padding for odd-length strings
func EncodeBCD(s string) ([]byte, error) {
    // Validate input is all digits
    for _, c := range s {
        if c < '0' || c > '9' {
            return nil, fmt.Errorf("invalid digit in BCD input: %c", c)
        }
    }

    // Left-pad to even length
    if len(s)%2 == 1 {
        s = "0" + s
    }

    result := make([]byte, len(s)/2)
    for i := 0; i < len(s); i += 2 {
        high := s[i] - '0'    // First digit → high nibble
        low := s[i+1] - '0'   // Second digit → low nibble
        result[i/2] = (high << 4) | low
    }

    return result, nil
}

// Examples:
// EncodeBCD("12345") → []byte{0x01, 0x23, 0x45}
// EncodeBCD("1234")  → []byte{0x12, 0x34}
// EncodeBCD("5")     → []byte{0x05}

Decoding BCD in Go

// DecodeBCD converts packed BCD bytes to a numeric string
// Returns the number with leading zeros preserved
func DecodeBCD(data []byte) string {
    var result strings.Builder
    result.Grow(len(data) * 2)

    for _, b := range data {
        high := (b >> 4) & 0x0F
        low := b & 0x0F

        // Each nibble must be 0-9 for valid BCD
        if high > 9 || low > 9 {
            // Invalid BCD - could return error or use placeholder
            result.WriteByte('?')
            result.WriteByte('?')
            continue
        }

        result.WriteByte('0' + high)
        result.WriteByte('0' + low)
    }

    return result.String()
}

// Examples:
// DecodeBCD([]byte{0x12, 0x34}) → "1234"
// DecodeBCD([]byte{0x01, 0x23, 0x45}) → "012345"

BCD Length Prefixes

Some implementations encode LLVAR/LLLVAR length prefixes in BCD:

LLVAR with BCD length:
Length 16 = 0x16 (single byte)

Wire: 16 34 35 33 32 30 31 35 31 31 32 38 33 30 33 36 36
      ├┤ └──────────────────────────────────────────────┘
     0x16         16 ASCII digits
     ↑
     BCD-encoded length = 16

This saves 1 byte per field compared to ASCII length prefix. Over millions of transactions, that adds up.

Length Prefix Ambiguity Rule

When bytes look ambiguous, do not infer prefix format by first nibble alone.

Prefer the field spec (it defines the contract).
Parse using the declared format (ASCII or BCD).
Check resulting cursor against message boundaries.
If both parse paths are possible, reject with a clear protocol error and log both interpretations.

// ParseLLVWithSpec parses LLVAR when the spec declares format.
func ParseLLVWithSpec(data []byte, offset int, asBCD bool, maxLen int) (int, error) {
    if asBCD {
        if offset >= len(data) {
            return 0, fmt.Errorf("not enough data for BCD length prefix")
        }
        b := data[offset]
        high := (b >> 4) & 0x0F
        low := b & 0x0F
        if high > 9 || low > 9 {
            return 0, fmt.Errorf("invalid BCD length byte: %02X", b)
        }
        length := int(high)*10 + int(low)
        if length > maxLen {
            return 0, fmt.Errorf("length %d exceeds max allowed %d", length, maxLen)
        }
        return length, nil
    }

    if offset+2 > len(data) {
        return 0, fmt.Errorf("not enough data for ASCII length prefix")
    }
    lengthStr := string(data[offset : offset+2])
    length, err := strconv.Atoi(lengthStr)
    if err != nil {
        return 0, fmt.Errorf("invalid ASCII length: %q", lengthStr)
    }
    if length > maxLen {
        return 0, fmt.Errorf("length %d exceeds max allowed %d", length, maxLen)
    }
    return length, nil
}

// ReadLLVARBCD reads LLVAR with BCD-encoded length prefix
func ReadLLVARBCD(data []byte, offset int) (string, int, error) {
    if offset >= len(data) {
        return "", offset, fmt.Errorf("no data for BCD length prefix")
    }

    // Single byte BCD length: 0x16 = 16
    b := data[offset]
    high := (b >> 4) & 0x0F
    low := b & 0x0F

    if high > 9 || low > 9 {
        return "", offset, fmt.Errorf("invalid BCD length byte: %02X", b)
    }

    length := int(high)*10 + int(low)

    valueStart := offset + 1  // Only 1 byte for BCD length
    if valueStart+length > len(data) {
        return "", offset, fmt.Errorf("value truncated")
    }

    value := string(data[valueStart : valueStart+length])
    return value, valueStart + length, nil
}

When to Use BCD

BCD appears in:

Numeric field values (some networks encode amounts in BCD)
Length prefixes (saves bytes)
Track data (mixed with other encodings)
Legacy systems (mainframes love BCD)

Modern systems typically use ASCII for readability and debugging, but BCD remains common in:

Visa BASE I/II
Mastercard MDS
Many Asian payment networks

ASCII vs EBCDIC

The Historical Context

In 1987, the world wasn't standardized on ASCII. IBM mainframes used EBCDIC (Extended Binary Coded Decimal Interchange Code), a different character encoding where 'A' is 0xC1, not 0x41.

ISO8583 was designed to work with both. Many bank mainframes still run EBCDIC internally.

Character Mapping Differences

Character  ASCII  EBCDIC
─────────  ─────  ──────
'0'        0x30   0xF0
'1'        0x31   0xF1
'9'        0x39   0xF9
'A'        0x41   0xC1
'Z'        0x5A   0xE9
'a'        0x61   0x81
' '        0x20   0x40

Detection Heuristic

How do you know which encoding a message uses?

// Heuristic: Look at what should be digits (0-9)
// In ASCII: 0x30-0x39
// In EBCDIC: 0xF0-0xF9

func DetectEncoding(data []byte) string {
    // Check first few bytes of what should be numeric data (like MTI)
    asciiDigits := 0
    ebcdicDigits := 0

    for _, b := range data[:4] {  // Check MTI
        if b >= 0x30 && b <= 0x39 {
            asciiDigits++
        }
        if b >= 0xF0 && b <= 0xF9 {
            ebcdicDigits++
        }
    }

    if asciiDigits >= 3 {
        return "ASCII"
    }
    if ebcdicDigits >= 3 {
        return "EBCDIC"
    }
    return "unknown"
}

EBCDIC Conversion

var ebcdicToASCII = [256]byte{
    // ... full 256-byte translation table ...
    // 0xF0 → 0x30 ('0')
    // 0xF1 → 0x31 ('1')
    // 0xC1 → 0x41 ('A')
    // etc.
}

func EBCDICToASCII(data []byte) []byte {
    result := make([]byte, len(data))
    for i, b := range data {
        result[i] = ebcdicToASCII[b]
    }
    return result
}

Practical Reality

In 2024, you'll mostly encounter ASCII. But if you're integrating with:

Legacy bank mainframes
Some older Visa/Mastercard endpoints
Regional networks in certain countries

...you might hit EBCDIC. When your parser produces garbage like "ðñòóô" instead of "01234", check for EBCDIC.

Padding Rules: Left vs Right, Zeros vs Spaces

Padding ensures fixed-length fields are always the right size. The rules vary by data type.

Numeric Fields: Right-Justified, Zero-Padded

Amount field (n 12):
Value $100.00 = 10000 cents

RIGHT:  000000010000  (leading zeros)
WRONG:  10000         (too short)
WRONG:  100000000000  (zero-padded on wrong side)
WRONG:  "  10000"     (spaces instead of zeros)

Why right-justified? Because the rightmost digit is the ones place. Leading zeros don't change the value.

func PadNumericRight(value string, length int) string {
    if len(value) >= length {
        return value[:length]  // Truncate if too long (or error?)
    }
    return strings.Repeat("0", length-len(value)) + value
}

// PadNumericRight("10000", 12) → "000000010000"

Alphanumeric Fields: Left-Justified, Space-Padded

Terminal ID field (ans 8):
Value "TERM01"

RIGHT:  "TERM01  "  (trailing spaces)
WRONG:  "  TERM01"  (leading spaces)
WRONG:  "TERM0100"  (zero-padded)

Why left-justified? Because trailing spaces are clearly padding. Leading spaces would look intentional.

func PadAlphanumericLeft(value string, length int) string {
    if len(value) >= length {
        return value[:length]
    }
    return value + strings.Repeat(" ", length-len(value))
}

// PadAlphanumericLeft("TERM01", 8) → "TERM01  "

Trimming When Reading

When reading padded fields, you typically want to strip the padding:

func TrimNumeric(value string) string {
    // Remove leading zeros, but keep at least one digit
    result := strings.TrimLeft(value, "0")
    if result == "" {
        return "0"
    }
    return result
}

func TrimAlphanumeric(value string) string {
    return strings.TrimRight(value, " ")
}

### Parsing-Safe Trimming

Trim rules are type-specific:

- Numeric: trim left padding only for display/reporting.
- Alphanumeric: trim right spaces only when the spec says space-padding.
- Binary: do not trim.

func TrimForDisplay(value, fieldType string) string { switch fieldType { case "n": value = strings.TrimLeft(value, "0") if value == "" { return "0" } return value case "an", "ans": return strings.TrimRight(value, " ") default: return value } }

The Track 2 Special Case

Track 2 data (DE35) has its own rules. It contains:

PAN (digits)
Separator (= or D)
Expiry date (YYMM)
Service code
Discretionary data

Track 2: "4532015112830366=2512101123400000"
         ├──────PAN───────┤ ├──────────────┤
                         Sep  Expiry + data

Padding: Right-padded with 'F' in BCD, or '=' padding in some specs

Track 2 encoding is complex enough that we'll dedicate a problem to it in Module 4.

Binary Fields

Binary fields contain raw bytes that shouldn't be interpreted as characters.

Common Binary Fields

DE52: PIN Block - b 8
Encrypted PIN data, exactly 8 bytes

DE55: EMV Data - LLLVAR b..999
TLV-encoded chip card data, variable length

DE64: MAC - b 8
Message Authentication Code, exactly 8 bytes

Handling Binary Data in Go

// Binary fields should be handled as []byte, not string
type Field struct {
    Number  int
    Type    string  // "string" or "binary"
    Value   string  // For character data
    Binary  []byte  // For binary data
}

func ReadBinaryField(data []byte, offset, length int) ([]byte, int, error) {
    if offset+length > len(data) {
        return nil, offset, fmt.Errorf("binary field truncated")
    }

    result := make([]byte, length)
    copy(result, data[offset:offset+length])

    return result, offset + length, nil
}

Displaying Binary Data

When debugging, you'll want to display binary fields in hex:

func FormatBinary(data []byte) string {
    var parts []string
    for _, b := range data {
        parts = append(parts, fmt.Sprintf("%02X", b))
    }
    return strings.Join(parts, " ")
}

// FormatBinary([]byte{0xA1, 0xB2, 0xC3}) → "A1 B2 C3"

Putting It Together: Field Specification Tables

In a real parser, you'd define a specification table mapping field numbers to their encoding rules:

type FieldSpec struct {
    Number     int
    Name       string
    LengthType string  // "FIXED", "LLVAR", "LLLVAR"
    MaxLength  int
    DataType   string  // "N", "AN", "ANS", "B"
    Encoding   string  // "ASCII", "BCD", "BINARY"
}

var ISO8583Spec = map[int]FieldSpec{
    2:  {2, "Primary Account Number", "LLVAR", 19, "N", "ASCII"},
    3:  {3, "Processing Code", "FIXED", 6, "N", "ASCII"},
    4:  {4, "Transaction Amount", "FIXED", 12, "N", "ASCII"},
    7:  {7, "Transmission Date/Time", "FIXED", 10, "N", "ASCII"},
    11: {11, "STAN", "FIXED", 6, "N", "ASCII"},
    12: {12, "Local Time", "FIXED", 6, "N", "ASCII"},
    13: {13, "Local Date", "FIXED", 4, "N", "ASCII"},
    14: {14, "Expiration Date", "FIXED", 4, "N", "ASCII"},
    22: {22, "POS Entry Mode", "FIXED", 3, "N", "ASCII"},
    23: {23, "Card Sequence Number", "FIXED", 3, "N", "ASCII"},
    25: {25, "POS Condition Code", "FIXED", 2, "N", "ASCII"},
    32: {32, "Acquiring Institution ID", "LLVAR", 11, "N", "ASCII"},
    35: {35, "Track 2 Data", "LLVAR", 37, "Z", "ASCII"},
    37: {37, "Retrieval Reference Number", "FIXED", 12, "AN", "ASCII"},
    38: {38, "Authorization Code", "FIXED", 6, "AN", "ASCII"},
    39: {39, "Response Code", "FIXED", 2, "AN", "ASCII"},
    41: {41, "Terminal ID", "FIXED", 8, "ANS", "ASCII"},
    42: {42, "Merchant ID", "FIXED", 15, "ANS", "ASCII"},
    43: {43, "Merchant Name/Location", "FIXED", 40, "ANS", "ASCII"},
    49: {49, "Currency Code", "FIXED", 3, "N", "ASCII"},
    52: {52, "PIN Data", "FIXED", 8, "B", "BINARY"},
    55: {55, "EMV Data", "LLLVAR", 999, "B", "BINARY"},
    // ... and so on for all 128+ fields
}

The Generic Field Reader

func ReadField(data []byte, offset int, spec FieldSpec) (interface{}, int, error) {
    switch spec.LengthType {
    case "FIXED":
        if spec.DataType == "B" {
            return ReadFixedBinary(data, offset, spec.MaxLength)
        }
        return ReadFixedField(data, offset, spec.MaxLength)

    case "LLVAR":
        return ReadLLVAR(data, offset)

    case "LLLVAR":
        return ReadLLLVAR(data, offset)

    default:
        return nil, offset, fmt.Errorf("unknown length type: %s", spec.LengthType)
    }
}

This is the pattern you'll see in production ISO8583 libraries.

Common Pitfalls

1. Length Includes Padding vs Length of Actual Data

Some implementations report the padded length, others the actual data length:

Field: ans 40, value "MERCHANT NAME"

Length option A: 13 (actual characters)
Length option B: 40 (padded field width)

For fixed fields, there's no ambiguity - always 40.
For LLVAR, the length should be actual data: 13

2. BCD with Odd Digits

When encoding a PAN like "4532015112830366" (16 digits, even) vs "4532015112830361" (still 16):

// Easy case: even digits
bcd := EncodeBCD("1234")  // → 0x12 0x34

// Odd case: where does the padding go?
bcd := EncodeBCD("123")   // → 0x01 0x23 (left-pad) or 0x12 0x30 (right-pad)?

Know your network's convention. Visa uses left-padding. Some Asian networks use right-padding.

3. Mixed Encoding in One Message

A message might have:

MTI in ASCII
Bitmap in binary
Amounts in BCD
Alphanumeric fields in ASCII
PIN block in binary

Don't assume uniform encoding. Parse each field according to its specification.

4. Length Prefix Encoding Mismatch

The length prefix and the value might use different encodings:

LLVAR with BCD length, ASCII value:
Length: 0x16 (BCD = 16)
Value:  31 36 (ASCII "16")

If you decode the length as ASCII, you'd read:
"16" = 0x31 0x36 = characters '1' and '6' = length 16?
No - those bytes as ASCII length would be 0x3136... garbage.

Check documentation for each network's length prefix encoding.

5. Right-to-Left Languages

Some fields (merchant name in Arabic/Hebrew) might contain right-to-left text. The bytes are still stored left-to-right, but display reverses. Don't let display order confuse your parsing.

Real-World Case Study: The Amount That Wasn't

A payment processor reported random transactions showing 10x the actual amount.

Investigation:

Expected amount: $125.50 → "000000012550" (12 chars)
Received amount: "00000001255" (11 chars... wait, that's wrong)
Parsed amount: $12,550.00 (missing trailing 0)

Root Cause: The POS terminal was encoding amounts in BCD to save bandwidth:

$125.50 = 12550 cents = 0x00 0x01 0x25 0x50 (4 bytes BCD)

But the receiving system expected ASCII:

0x00 0x01 0x25 0x50 read as ASCII = "\x00\x01%P" (garbage)

The parser had fallback logic that stripped non-digits and interpreted what remained, producing wildly wrong values.

The fix: Detect BCD vs ASCII encoding in the amount field. If the first byte is 0x00-0x09 (not 0x30-0x39), it's BCD.

Lesson: Encoding mismatches don't always crash. Sometimes they silently corrupt data. That's worse.

Summary

Field encoding in ISO8583 is about knowing three things for every field:

How long is it? (Fixed N bytes, or LLVAR/LLLVAR)
What characters are valid? (Numeric, alphanumeric, binary)
How are bytes encoded? (ASCII, BCD, EBCDIC, raw binary)

Key concepts:

Aspect	Options	Notes
Length	Fixed, LLVAR, LLLVAR, LLLLVAR	LLVAR = 2-char prefix
Data Type	n, a, an, ans, b, z	n=numeric, b=binary
Encoding	ASCII, BCD, EBCDIC	BCD packs 2 digits/byte
Padding	Zero (numeric), Space (alpha)	Right-justify numbers

The critical skill: When parsing fails, identify which of these three things you got wrong. Usually it's:

Wrong length type (LLVAR vs LLLVAR)
Wrong encoding (ASCII vs BCD)
Wrong padding (stripping when you shouldn't, or vice versa)

What's Next

Now that you understand field encoding, you're ready to:

bcd-codec: Implement BCD encoding and decoding
llvar-codec: Implement LLVAR/LLLVAR field reading and writing
field-encoder: Build a generic field encoder using specifications

These foundational encoders will be used in every subsequent module when parsing real messages.

Module Items

BCD Codec
ISO8583 problem: BCD Codec
easy iso8583 · payments
LLVAR Codec
ISO8583 problem: LLVAR Codec
medium Sign in to access medium and hard problems
Field Encoder
ISO8583 problem: Field Encoder
medium Sign in to access medium and hard problems
Field Encoding Types Quiz
Test your understanding of ISO8583 field encoding: fixed vs variable length, BCD encoding, padding rules, and data type validation.
Quiz

Lesson

Module 3: Field Encoding Types

The Encoding Problem

Understanding Field Specifications

Fixed-Length Fields

Numeric Fixed Fields (n)

Alphanumeric Fixed Fields (an, ans)

Binary Fixed Fields (b)

The Go Implementation

Variable-Length Fields: LLVAR and LLLVAR

LLVAR: 2-Digit Length Prefix

LLLVAR: 3-Digit Length Prefix

LLLLVAR: 4-Digit Length Prefix (Extension)

ASCII vs BCD Length Prefixes

The Go Implementation

Common Bugs with Variable-Length Fields

BCD Encoding: Packing Digits

The Basic Idea

Odd-Length Padding

Encoding BCD in Go

Decoding BCD in Go

BCD Length Prefixes

Length Prefix Ambiguity Rule

When to Use BCD

ASCII vs EBCDIC

The Historical Context

Character Mapping Differences

Detection Heuristic

EBCDIC Conversion

Practical Reality

Padding Rules: Left vs Right, Zeros vs Spaces

Numeric Fields: Right-Justified, Zero-Padded

Alphanumeric Fields: Left-Justified, Space-Padded

Trimming When Reading

The Track 2 Special Case

Binary Fields

Common Binary Fields

Handling Binary Data in Go

Displaying Binary Data

Putting It Together: Field Specification Tables

The Generic Field Reader

Common Pitfalls

1. Length Includes Padding vs Length of Actual Data

2. BCD with Odd Digits

3. Mixed Encoding in One Message

4. Length Prefix Encoding Mismatch

5. Right-to-Left Languages

Real-World Case Study: The Amount That Wasn't

Summary

What's Next

Module Items

BCD Codec

LLVAR Codec

Field Encoder

Field Encoding Types Quiz