The migration engine, from F-table extraction to reconciler

A reference architecture for the JDE-to-D365 migration engine — extractor, canonical model, transformer, loader, reconciler — built around the F-table families that matter, the six JDE-specific normalisation problems that break a first attempt, the canonical lookup registry, transformation-matrix governance, Data Management Framework sequencing, dry-run mode, and the fifteen-control operational-hardening register that makes the engine trustworthy enough for a live cutover.

What you will be able to do

Introduction

Chapter 1 set the programme's shape: an estate inventory, an account-key decomposition decision, and a rollout shape chosen against JDE's usual multi-company, single-instance footprint.

The reference architecture: from extractor to reconciler

A representative reference implementation for a mid-size JDE estate runs to somewhere around 29 canonical concepts and a transformation matrix covering 13 business objects and roughly 640 individual field mappings — smaller in absolute count than a deeply hierarchical source system, because JDE's own organisational model is comparatively…

The extractor: reading JDE directly, and the F-table families that matter

JDE's single most useful architectural fact for a migration programme is this: the F-tables are ordinary relational tables in an ordinary database — Oracle Database, Microsoft SQL Server, or Db2 for i, depending on how the estate is hosted — and many JDE sites already grant direct, read-only SQL access to them for reporting.

Six JDE-specific normalisation problems

None of the following are exotic edge cases. Each one is encountered on essentially every JDE extraction, and each one has caused a real reconciliation failure somewhere.

The canonical lookup registry

JDE's complexity is not organisational depth — Company and Business Unit are comparatively few levels — it is the density of small, locally meaningful codes layered on top: AAI items, ledger types, category codes, UDCs and subledger types, each translated slightly differently depending on who last touched the mapping.

Transformation-matrix governance: who signs off each rule

Where the canonical model governs values, the transformation matrix governs fields. Each business object in scope — Address Book, Item Master, GL Balances, AR open items, AP open items, Fixed Assets, Sales Orders, Purchase Orders and Job Cost among them — has its own versioned matrix: source table and field, source type, target entity…

Loading through the Data Management Framework

Every entity imported through the Data Management Framework runs the same three-phase pipeline: Source to Staging (file parsing and column-mapping validation), Staging to Data Entity (business logic, duplicate detection, foreign-key validation against the current database state), and Entity to Physical Tables (a set-based write through…

Dry-run mode and repeatability

Dry-run mode runs the entire pipeline — extraction through the canonical registry, the transformation matrix, and DMF package assembly — against fixture data shaped like the real JDE estate's edge cases, with no write to F&O at all, and checks the resulting output against a known-correct expectation.

Operational hardening: the fifteen-control register

Fifteen specific controls separate a working prototype from an engine trustworthy enough to run a live cutover. Each closes one concrete risk.

Knowledge check

Summary

A migration engine earns the word "engine" through architecture and operational discipline, not through the number of scripts it contains.