Profile boundary: The worked messages and code below use explicit teaching profiles. Field widths, mandatory fields, response actions, timers, and matching keys must come from the selected exercise or a versioned endpoint agreement. Schematic dumps and scenarios are synthetic; they are not network certification fixtures. Diagnostic snippets illustrate mechanisms and are not a complete production library.
ISO8583 has 128 possible fields. No message uses all of them.
How do you know which fields are present?
Option A: Label each field
┌───────────────────────────────────────────────────────────────────┐
│ "Field02:5412345678901234,Field03:000000,Field04:000000010000" │
│ │
│ 7+ bytes overhead per field × 20 fields = 140+ bytes wasted │
└───────────────────────────────────────────────────────────────────┘
Option B: Bitmap (ISO8583's solution)
┌───────────────────────────────────────────────────────────────────┐
│ 8 bytes → indicates presence of up to 64 fields │
│ 72 3A 44 81 08 E1 80 00 → fields 2,3,4,7,11,12,13,15,... │
└───────────────────────────────────────────────────────────────────┘
8 bytes vs 140 bytes. That's why ISO8583 uses bitmaps.
Every ISO8583 message has this structure:
┌──────────┬─────────────────────────┬────────────────────────────────┐
│ MTI │ PRIMARY BITMAP │ DATA ELEMENTS │
│ 4 chars │ 8 bytes │ (variable length) │
│ │ (always) │ (parsed in order) │
└──────────┴─────────────────────────┴────────────────────────────────┘
│ │ │
│ │ └─► Only fields with
│ │ bits set in bitmap
│ │
│ └─► 64 bits = 64 possible fields (1-64)
│
└─► Already parsed (Module 1)
The primary bitmap has 64 decoded bits; binary and hex-ASCII encodings occupy 8 and 16 wire bytes respectively.
ISO8583 uses 1-BASED numbering with MSB-FIRST ordering
Byte 0: 72 (hex) = 0111 0010 (binary)
┌─ Bit 1 (MSB)
│
▼
0 1 1 1 0 0 1 0
│ │ │ │ │ │ │ │
Bit #: 1 2 3 4 5 6 7 8
Fields present: 2, 3, 4, 7
CRITICAL: Bit 1 is LEFTMOST, not rightmost!
CRITICAL: Field numbers start at 1, not 0!
To check if field N is present:
┌─────────────────────────────────────────────────────────────┐
│ │
│ byteIndex = (N - 1) / 8 │
│ bitPosition = 7 - ((N - 1) % 8) │
│ isPresent = (bitmap[byteIndex] >> bitPosition) & 1 │
│ │
└─────────────────────────────────────────────────────────────┘
Example: Is field 11 present?
byteIndex = (11 - 1) / 8 = 10 / 8 = 1
bitPosition = 7 - ((11 - 1) % 8) = 7 - 2 = 5
Check: bit 5 of byte 1
Byte 1: 3A = 0011 1010
│
└─ position 5 (counting from right, 0-indexed)
Value = 1 → Field 11 IS present ✓
| Field | byteIndex | bitPosition |
|---|---|---|
| 1 | 0 | 7 |
| 8 | 0 | 0 |
| 9 | 1 | 7 |
| 16 | 1 | 0 |
| 64 | 7 | 0 |
| 65 | 8 | 7 |
| 128 | 15 | 0 |
If these are wrong, every later lookup will be wrong.
Bitmap: 72 3A 44 81 08 E1 80 00
Byte 0: 72 = 0111 0010 Fields: 2, 3, 4, 7
Byte 1: 3A = 0011 1010 Fields: 11, 12, 13, 15
Byte 2: 44 = 0100 0100 Fields: 18, 22
Byte 3: 81 = 1000 0001 Fields: 25, 32
Byte 4: 08 = 0000 1000 Fields: 37
Byte 5: E1 = 1110 0001 Fields: 41, 42, 43, 48
Byte 6: 80 = 1000 0000 Fields: 49
Byte 7: 00 = 0000 0000 Fields: (none)
Total present: 2, 3, 4, 7, 11, 12, 13, 15, 18, 22, 25, 32, 37, 41, 42, 43, 48, 49
Typical authorization request fields!
┌──────────────────────────────────────────────────────────────────┐
│ Bit 1 = 0: Primary bitmap only (fields 1-64) │
│ │
│ ┌──────────┬────────────────────┬───────────────────┐ │
│ │ MTI │ Primary Bitmap │ Data Elements │ │
│ │ │ 8 bytes │ │ │
│ └──────────┴────────────────────┴───────────────────┘ │
├──────────────────────────────────────────────────────────────────┤
│ Bit 1 = 1: Primary + Secondary bitmap (fields 1-128) │
│ │
│ ┌──────────┬────────────────┬────────────────────┬───────────┐ │
│ │ MTI │ Primary Bitmap │ Secondary Bitmap │ Data │ │
│ │ │ 8 bytes,bit1=1 │ 8 bytes (= DE1) │ Elements │ │
│ └──────────┴────────────────┴────────────────────┴───────────┘ │
└──────────────────────────────────────────────────────────────────┘
IMPORTANT: The secondary bitmap IS Field 1 (DE1)
There is no separate "Field 1 data"
When Bit 1 = 1:
Primary Bitmap Secondary Bitmap
┌────────────────┐ ┌────────────────┐
│ F2 34 00 01... │ │ 80 00 00 00... │
│ (Bit 1 = 1) │ │ │
└────────────────┘ └────────────────┘
│ │
│ └─► Covers fields 65-128
│ Bit 1 here = field 65
│ Bit 2 here = field 66
│
└─► Covers fields 1-64 (as before)
Bit 1 = 1 means "secondary follows"
Bit 2 = field 2
...
Fields 65-128 only possible when primary bit 1 = 1
ISO 8583:2003 extends to fields 129-192
Pattern continues:
┌─────────────────────────────────────────────────────────────────┐
│ Bit 1 of Primary (field 1) → Secondary bitmap follows │
│ Bit 1 of Secondary (field 65) → Tertiary bitmap follows │
└─────────────────────────────────────────────────────────────────┘
Full structure:
┌──────┬───────────┬───────────────┬───────────────┬──────────────┐
│ MTI │ Primary │ Secondary │ Tertiary │ Data │
│ │ 8 bytes │ 8 bytes │ 8 bytes │ Elements │
│ │ bit1 = 1 │ bit1 = 1 │ (= DE65) │ │
└──────┴───────────┴───────────────┴───────────────┴──────────────┘
24 bytes of bitmap = 192 possible fields
In practice: Tertiary bitmaps are rare. Most systems use 1987 spec.
The SAME bitmap can be transmitted two ways:
Binary (8 bytes):
┌──────────────────────────────────────────────────────────┐
│ 72 3A 44 81 08 E1 80 00 │
│ (raw byte values) │
└──────────────────────────────────────────────────────────┘
Hex-ASCII (16 characters = 16 bytes):
┌──────────────────────────────────────────────────────────┐
│ "723A448108E18000" │
│ 37 32 33 41 34 34 38 31 30 38 45 31 38 30 30 30 │
│ '7''2''3''A''4''4''8''1''0''8''E''1''8''0''0''0' │
└──────────────────────────────────────────────────────────┘
HOW TO DETECT:
- 8 bytes → Binary
- 16 bytes with values 0x30-0x39, 0x41-0x46 → Hex-ASCII
When parsing fails: CHECK ENCODING FIRST
When unsure of encoding:
// ParseBitmaps reads only what the control bits demand.
func ParseBitmaps(input []byte, isHex bool, supportTertiary bool) (primary, secondary, tertiary []byte, consumed int, err error) {
const (
BIN_LEN = 8
HEX_LEN = 16
)
baseLen := BIN_LEN
if isHex { baseLen = HEX_LEN }
if len(input) < baseLen {
return nil, nil, nil, 0, ErrBitmapTooShort
}
// Decode primary bitmap from input[:baseLen]
primary = append([]byte(nil), input[:baseLen]...)
consumed = baseLen
// If primary bit 1 is set, decode secondary
if IsFieldPresent(primary, 1) {
if len(input) < consumed+baseLen {
return nil, nil, nil, 0, ErrBitmapTooShort
}
secondary = append([]byte(nil), input[consumed:consumed+baseLen]...)
consumed += baseLen
// If secondary bit 1 is set, decode tertiary (tertiary support)
if supportTertiary && IsFieldPresent(secondary, 1) {
if len(input) < consumed+baseLen {
return nil, nil, nil, 0, ErrBitmapTooShort
}
tertiary = append([]byte(nil), input[consumed:consumed+baseLen]...)
consumed += baseLen
}
}
return primary, secondary, tertiary, consumed, nil
}
The rule is simple:
// IsFieldPresent checks if field N is present in bitmap
func IsFieldPresent(bitmap []byte, field int) bool {
if field < 1 || field > len(bitmap)*8 {
return false
}
byteIndex := (field - 1) / 8
bitPosition := 7 - ((field - 1) % 8)
return (bitmap[byteIndex] >> bitPosition) & 1 == 1
}
// Example usage:
bitmap := []byte{0x72, 0x3A, 0x44, 0x81, 0x08, 0xE1, 0x80, 0x00}
IsFieldPresent(bitmap, 2) // true (PAN)
IsFieldPresent(bitmap, 5) // false
IsFieldPresent(bitmap, 11) // true (STAN)
// fieldsFromBitmap and bitmapFromFields should round-trip.
func fieldsFromBitmap(bitmap []byte) []int {
out := make([]int, 0)
for field := 1; field <= len(bitmap)*8; field++ {
if IsFieldPresent(bitmap, field) {
out = append(out, field)
}
}
return out
}
func bitmapRoundTripInvariant(fields []int) bool {
a := buildBitmapFromFields(fields)
b := buildBitmapFromFields(fieldsFromBitmap(a))
return bytes.Equal(a, b)
}
// Property-based test idea:
// generate random field lists -> invariant must hold
// GetPresentFields returns all field numbers with bit set
func GetPresentFields(bitmap []byte) []int {
var fields []int
for field := 1; field <= len(bitmap)*8; field++ {
byteIndex := (field - 1) / 8
bitPosition := 7 - ((field - 1) % 8)
if (bitmap[byteIndex] >> bitPosition) & 1 == 1 {
fields = append(fields, field)
}
}
return fields
}
// Example:
bitmap := []byte{0x72, 0x3A} // First 16 bits only
fields := GetPresentFields(bitmap)
// Result: []int{2, 3, 4, 7, 11, 12, 13, 15}
// BuildBitmap creates bitmap from field list
func BuildBitmap(fields []int) []byte {
// Find max field to determine bitmap size
maxField := 0
for _, f := range fields {
if f > maxField {
maxField = f
}
}
size := 8 // Primary
if maxField > 64 {
size = 16 // + Secondary
}
bitmap := make([]byte, size)
for _, field := range fields {
byteIndex := (field - 1) / 8
bitPosition := 7 - ((field - 1) % 8)
bitmap[byteIndex] |= (1 << bitPosition)
}
// Set bit 1 if secondary exists
if size > 8 {
bitmap[0] |= 0x80
}
return bitmap
}
┌──────────────────────────────────────────────────────────────────┐
│ 1. OFF-BY-ONE ERROR │
│ WRONG: byteIndex = field / 8 │
│ RIGHT: byteIndex = (field - 1) / 8 │
├──────────────────────────────────────────────────────────────────┤
│ 2. BIT ORDER CONFUSION │
│ WRONG: Bit 1 is rightmost (LSB) │
│ RIGHT: Bit 1 is leftmost (MSB) │
├──────────────────────────────────────────────────────────────────┤
│ 3. FORGETTING BIT 1 WHEN BUILDING │
│ WRONG: Just set field 70's bit │
│ RIGHT: Set bit 1 FIRST, then field 70 │
├──────────────────────────────────────────────────────────────────┤
│ 4. ENCODING MISMATCH │
│ WRONG: Assume 8 bytes │
│ RIGHT: Check if 16 hex-ASCII chars │
├──────────────────────────────────────────────────────────────────┤
│ 5. BITMAP/DATA MISMATCH │
│ If bitmap says 10 fields but data has 11: │
│ Every field after the extra is misaligned → garbage │
└──────────────────────────────────────────────────────────────────┘
PRINT THE BITMAP IN BINARY:
func PrintBitmap(bitmap []byte) {
for i, b := range bitmap {
for bit := 7; bit >= 0; bit-- {
field := i*8 + (7 - bit) + 1
if (b >> bit) & 1 == 1 {
fmt.Printf("Field %2d: PRESENT\n", field)
}
}
}
}
Output:
Field 2: PRESENT
Field 3: PRESENT
Field 4: PRESENT
Field 7: PRESENT
Field 11: PRESENT
...
Compare expected vs actual fields to find mismatches!
1) Parse stage
- Confirm expected encoding: binary(8/16 bytes) vs hex-ASCII(16/32 chars)
- Validate length and allowed chars before field parsing
2) Indicator stage
- If primary bit 1 = 1 -> secondary must be present
- If secondary bit 1 = 1 -> check the tertiary extension support contract
3) Presence stage
- Compare bitmap-derived fields vs expected message fields
- Any unexpected set/missing bit is a protocol mismatch
4) Alignment stage
- Parse first few fields in lockstep and verify cursor movement
- One wrong length => all subsequent fields misaligned
5) Recovery
- Log MTI + bitmap bytes + expected/actual field diff
- Keep it reproducible before changing decode logic
For high-throughput systems (100K+ TPS):
BASIC: Array indexing
bitmap[byteIndex] >> bitPosition & 1
~2ns per check
FAST: Use uint64 for primary bitmap
bitmap := binary.BigEndian.Uint64(data)
(bitmap >> (64 - field)) & 1
~0.5ns per check
FASTEST: Precomputed masks
var masks [65]uint64 // masks[n] checks field n
bitmap & masks[field] != 0
~0.3ns per check
For most applications: Basic is fine.
For network switches: Every nanosecond matters.
┌────────────────────────────────────────────────────────────────┐
│ BITMAP QUICK REFERENCE │
├────────────────────────────────────────────────────────────────┤
│ Primary Bitmap: 8 bytes, fields 1-64 │
│ Secondary Bitmap: 8 bytes, fields 65-128 (when bit1=1) │
│ Tertiary Bitmap: 8 bytes, fields 129-192 (2003 spec) │
├────────────────────────────────────────────────────────────────┤
│ Bit Numbering: 1-based, MSB-first, big-endian │
│ │
│ Field N present? byteIdx = (N-1)/8 │
│ bitPos = 7 - ((N-1)%8) │
│ check: (bitmap[byteIdx] >> bitPos) & 1 │
├────────────────────────────────────────────────────────────────┤
│ Set field N: bitmap[byteIdx] |= (1 << bitPos) │
│ Clear field N: bitmap[byteIdx] &= ^(1 << bitPos) │
├────────────────────────────────────────────────────────────────┤
│ Bit 1 = 1: Secondary bitmap follows │
│ Bit 65 = 1: Tertiary bitmap follows (2003 only) │
├────────────────────────────────────────────────────────────────┤
│ Common fields: │
│ 2=PAN, 3=ProcCode, 4=Amount, 11=STAN, 39=RespCode │
│ 64=MAC, 70=NetMgmt, 95=Replacement, 128=Private use │
└────────────────────────────────────────────────────────────────┘
1. BITMAP = TABLE OF CONTENTS
└─► Tells you which fields are present before parsing data
2. 8 BYTES = 64 FIELDS
└─► Compact: 8 bytes covers 64 fields vs 100+ bytes of labels
3. BIT 1 IS SPECIAL
└─► When set, secondary bitmap follows (for fields 65-128)
4. 1-BASED, MSB-FIRST
└─► Field 1 = leftmost bit of first byte
└─► Not 0-based! This causes bugs.
5. THE FORMULA
└─► byteIndex = (field-1)/8
└─► bitPosition = 7 - ((field-1)%8)
6. WATCH FOR ENCODING
└─► Binary (8 bytes) vs Hex-ASCII (16 chars)
└─► When parsing fails, check this first!