Module 3: Field Encoding Types - Slides
1 / 21
After parsing the bitmap, you know which fields are present.
Now you need to know how to read them.
┌─────────────────────────────────────────────────────────────┐
│ Field What You Know What You Need to Know │
│ ───── ────────────── ──────────────────────── │
│ DE2 "It's present" → LLVAR, max 19, numeric │
│ DE3 "It's present" → Fixed 6, numeric │
│ DE4 "It's present" → Fixed 12, numeric │
│ DE52 "It's present" → Fixed 8, binary │
└─────────────────────────────────────────────────────────────┘
Every field has a specification:
ISO8583 uses a compact notation:
n 6 Fixed numeric, 6 characters
n..19 Variable numeric, up to 19 (LLVAR)
an 12 Fixed alphanumeric, 12 characters
ans..40 Variable alphanum+special, up to 40 (LLVAR)
b 8 Fixed binary, 8 bytes
b...999 Variable binary, up to 999 (LLLVAR)
Data type prefixes:
n = Numeric only (0-9)
a = Alphabetic only (A-Z, a-z)
an = Alphanumeric (letters + digits)
ans = Alphanumeric + Special (printable ASCII)
b = Binary (raw bytes)
z = Track 2 character set
Length suffixes:
6 = Exactly 6 (fixed)
..99 = Up to 99 (2-digit prefix = LLVAR)
...999 = Up to 999 (3-digit prefix = LLLVAR)
Always exactly N bytes. No length prefix needed.
DE3: Processing Code (n 6)
┌────────────────────────────┐
│ 30 30 30 30 30 30 │ ← Always 6 bytes
│ "0 0 0 0 0 0" │
└────────────────────────────┘
DE4: Amount (n 12)
┌────────────────────────────────────────────────┐
│ 30 30 30 30 30 30 30 31 30 30 30 30 │ ← Always 12 bytes
│ "0 0 0 0 0 0 0 1 0 0 0 0" │
└────────────────────────────────────────────────┘
Value: $100.00 (cents)
Reading fixed fields:
func ReadFixed(data []byte, offset, length int) string {
return string(data[offset : offset+length])
}
Problem: How does the parser know where DE2 (PAN) ends?
Solution: 2-character length prefix
DE2: PAN (LLVAR n..19)
Wire bytes:
┌────┬──────────────────────────────────────────────────┐
│ 16 │ 4 5 3 2 0 1 5 1 1 2 8 3 0 3 6 6 │
├────┼──────────────────────────────────────────────────┤
│"16"│ "4532015112830366" │
└────┴──────────────────────────────────────────────────┘
↑ ↑
Length prefix 16 characters of PAN
(ASCII "16")
Reading LLVAR:
func ReadLLVAR(data []byte, offset int) (string, int) {
length, _ := strconv.Atoi(string(data[offset:offset+2]))
value := string(data[offset+2 : offset+2+length])
return value, offset + 2 + length
}
LLLVAR: 3-character length prefix, supports up to 999 bytes
DE43: Merchant Name/Location (LLLVAR ans...40)
Wire:
┌──────┬─────────────────────────────────────────────┐
│ 035 │ A C M E S T O R E 1 2 3 M A I N ... │
├──────┼─────────────────────────────────────────────┤
│"035" │ 35 characters of data │
└──────┴─────────────────────────────────────────────┘
↑
3-digit length prefix
Pattern:
LLVAR: 2-digit prefix → max 99 bytes
LLLVAR: 3-digit prefix → max 999 bytes
LLLLVAR: 4-digit prefix → max 9999 bytes (rare)
BCD (Binary-Coded Decimal): Pack 2 digits into 1 byte
ASCII: Each digit = 1 byte
"12345" = 31 32 33 34 35 (5 bytes)
BCD: Each digit = 4 bits (nibble)
"12345" = 01 23 45 (3 bytes, left-padded)
Digit → Nibble
0 → 0000 5 → 0101
1 → 0001 6 → 0110
2 → 0010 7 → 0111
3 → 0011 8 → 1000
4 → 0100 9 → 1001
Example decode:
Byte: 0x47
Binary: 0100 0111
↓ ↓
4 7
Result: "47"
// Encode string to BCD (left-pad odd lengths)
func EncodeBCD(s string) []byte {
if len(s)%2 == 1 {
s = "0" + s // Left-pad to even length
}
result := make([]byte, len(s)/2)
for i := 0; i < len(s); i += 2 {
high := s[i] - '0'
low := s[i+1] - '0'
result[i/2] = (high << 4) | low
}
return result
}
// Decode BCD to string
func DecodeBCD(data []byte) string {
var result strings.Builder
for _, b := range data {
result.WriteByte('0' + (b >> 4)) // High nibble
result.WriteByte('0' + (b & 0x0F)) // Low nibble
}
return result.String()
}
Examples:
EncodeBCD("1234") → []byte{0x12, 0x34}
EncodeBCD("123") → []byte{0x01, 0x23} // Left-padded
DecodeBCD([]byte{0x12, 0x34}) → "1234"
Some networks encode LLVAR lengths in BCD (saves 1 byte):
ASCII length prefix (standard):
┌────┬────────────────┐
│"16"│ 16 bytes data │ Length = 2 bytes (0x31, 0x36)
└────┴────────────────┘
BCD length prefix:
┌────┬────────────────┐
│0x16│ 16 bytes data │ Length = 1 byte (0x16 = 16)
└────┴────────────────┘
Detection:
// ASCII prefix: bytes are 0x30-0x39 ('0'-'9')
// BCD prefix: bytes are 0x00-0x99 (raw BCD values)
if data[offset] >= 0x30 && data[offset] <= 0x39 {
// ASCII length prefix
} else {
// Likely BCD length prefix
}
Do this before parsing:
Failing fast here prevents deep parser drift.
Two character encodings, same data:
ASCII EBCDIC
'0' 0x30 0xF0
'1' 0x31 0xF1
'9' 0x39 0xF9
'A' 0x41 0xC1
'Z' 0x5A 0xE9
' ' 0x20 0x40
Detection heuristic:
// Check if first byte of MTI looks like ASCII or EBCDIC digit
if data[0] >= 0x30 && data[0] <= 0x39 {
return "ASCII" // '0'-'9'
}
if data[0] >= 0xF0 && data[0] <= 0xF9 {
return "EBCDIC" // EBCDIC '0'-'9'
}
When you see it: Legacy bank mainframes, some regional networks.
Symptom: Parser produces garbage like "ðñòó" instead of "0123".
Numeric fields: Right-justified, zero-padded
Field: n 12 (Amount)
Value: 10000 (cents = $100.00)
CORRECT: 0 0 0 0 0 0 0 1 0 0 0 0
└─leading zeros─┘ └value┘
WRONG: 1 0 0 0 0 0 0 0 0 0 0 0 (zeros on wrong side)
WRONG: _ _ _ _ _ _ _ 1 0 0 0 0 (spaces not zeros)
WRONG: 1 0 0 0 0 (not padded to length)
Go implementation:
func PadNumeric(value string, length int) string {
return strings.Repeat("0", length-len(value)) + value
}
PadNumeric("10000", 12) → "000000010000"
Alphanumeric fields: Left-justified, space-padded
Field: ans 8 (Terminal ID)
Value: "TERM01"
CORRECT: T E R M 0 1 _ _
└─value──┘ └pad┘
WRONG: _ _ T E R M 0 1 (spaces on wrong side)
WRONG: T E R M 0 1 0 0 (zeros not spaces)
Go implementation:
func PadAlphanumeric(value string, length int) string {
return fmt.Sprintf("%-*s", length, value)
// Or: value + strings.Repeat(" ", length-len(value))
}
PadAlphanumeric("TERM01", 8) → "TERM01 "
func TrimForDisplay(value, fieldType string) string {
switch fieldType {
case "n":
value = strings.TrimLeft(value, "0")
if value == "" {
return "0"
}
return value
case "an", "ans":
return strings.TrimRight(value, " ")
default:
return value
}
}
Why:
Binary fields: Raw bytes, no character encoding
DE52: PIN Data (b 8)
Wire: A1 B2 C3 D4 E5 F6 A7 B8
NOT "A1B2C3D4E5F6A7B8" (that would be 16 ASCII chars)
But actual bytes: [0xA1, 0xB2, 0xC3, 0xD4, 0xE5, 0xF6, 0xA7, 0xB8]
Reading binary fields:
func ReadBinaryField(data []byte, offset, length int) []byte {
result := make([]byte, length)
copy(result, data[offset:offset+length])
return result
}
Displaying for debugging:
func FormatHex(data []byte) string {
var parts []string
for _, b := range data {
parts = append(parts, fmt.Sprintf("%02X", b))
}
return strings.Join(parts, " ")
}
// [0xA1, 0xB2] → "A1 B2"
Define once, use everywhere:
type FieldSpec struct {
Number int
Name string
LengthType string // "FIXED", "LLVAR", "LLLVAR"
MaxLength int
DataType string // "N", "AN", "ANS", "B"
}
var Specs = map[int]FieldSpec{
2: {2, "PAN", "LLVAR", 19, "N"},
3: {3, "Processing Code","FIXED", 6, "N"},
4: {4, "Amount", "FIXED", 12, "N"},
11: {11, "STAN", "FIXED", 6, "N"},
35: {35, "Track 2", "LLVAR", 37, "Z"},
41: {41, "Terminal ID", "FIXED", 8, "ANS"},
52: {52, "PIN Data", "FIXED", 8, "B"},
// ... 120+ more fields
}
func ReadField(data []byte, offset int, spec FieldSpec) (string, int, error) {
switch spec.LengthType {
case "FIXED":
if spec.DataType == "B" {
bin, newOff, err := ReadBinaryField(data, offset, spec.MaxLength)
return FormatHex(bin), newOff, err
}
return ReadFixedField(data, offset, spec.MaxLength)
case "LLVAR":
return ReadLLVAR(data, offset)
case "LLLVAR":
return ReadLLLVAR(data, offset)
}
return "", offset, fmt.Errorf("unknown length type")
}
Usage:
spec := Specs[2] // PAN spec
value, newOffset, err := ReadField(data, offset, spec)
// value = "4532015112830366"
1. Length prefix included in value:
// WRONG
length := 16
value := data[offset:offset+length] // Includes "16" prefix!
// RIGHT
length := parseLength(data[offset:offset+2])
value := data[offset+2 : offset+2+length]
2. Wrong padding direction:
// Amount $100.00 in n 12 field
PadLeft("10000", 12, '0') // ✓ "000000010000"
PadRight("10000", 12, '0') // ✗ "100000000000"
3. BCD odd-length handling:
// "123" → BCD
LeftPad: 0x01 0x23 // ✓ Standard
RightPad: 0x12 0x30 // ✗ Some networks
// Know your network's convention!
┌─────────────────────────────────────────────────────────────┐
│ How to Encode This Field? │
└─────────────────────────────────────────────────────────────┘
│
Check LengthType
│
┌──────────────────┼──────────────────┐
│ │ │
FIXED LLVAR LLLVAR
│ │ │
Read N bytes Read 2-byte len Read 3-byte len
│ │ │
└──────────────────┴──────────────────┘
│
Check DataType
│
┌──────────────────┼──────────────────┐
│ │ │
N/AN/ANS B Z
│ │ │
Decode as text Keep as []byte Track 2 rules
│
Check Encoding
│
┌────┴────┐
ASCII BCD
│ │
Direct Unpack digits
┌─────────────────────────────────────────────────────────────┐
│ FIELD ENCODING QUICK REFERENCE │
├─────────────────────────────────────────────────────────────┤
│ LENGTH TYPES │
│ FIXED Read exactly N bytes │
│ LLVAR 2-char prefix (max 99) │
│ LLLVAR 3-char prefix (max 999) │
│ │
│ DATA TYPES │
│ n Numeric (0-9) │
│ an Alphanumeric (A-Z, a-z, 0-9) │
│ ans Alphanumeric + Special │
│ b Binary (raw bytes) │
│ │
│ PADDING │
│ Numeric: Right-justified, zero-padded │
│ Alpha: Left-justified, space-padded │
│ │
│ BCD │
│ Encode: 2 digits → 1 byte (high nibble, low nibble) │
│ Decode: 1 byte → 2 digits │
│ Odd length: Left-pad with '0' before encoding │
│ │
│ DEBUGGING │
│ 1. Check length type (LLVAR vs FIXED) │
│ 2. Check encoding (ASCII vs BCD) │
│ 3. Check padding direction │
│ 4. Verify length prefix encoding (ASCII vs BCD) │
└─────────────────────────────────────────────────────────────┘
Next: Implement BCD codec, LLVAR codec, and field encoder!