Why pre‑Migration testing matters

Upgrading or migrating a ColdFusion (CFML) application—whether to a newer Adobe ColdFusion release (e.g., 2018 → 2021 → 2023), moving from Adobe ColdFusion (ACF) to Lucee, or modernizing your runtime/JDK—unlocks Security patches, Performance improvements, new language Features, and supportability. Yet CFML applications often interact with legacy databases, PDF/Imaging toolchains, Search (Solr/Elasticsearch), and web server connectors (IIS/Apache/Tomcat/AJP), which makes regressions likely if you don’t thoroughly test ahead of time. A disciplined, repeatable pre‑Migration testing process reduces risk, shortens cutover windows, and accelerates successful go‑lives.

---

Prerequisites / Before You Start

• Inventory and documentation

  • List the current ColdFusion version and edition (Standard/Enterprise) and any cumulative hotfixes or updates.
  • Record OS, container or VM images, JDK/JRE version, and web server (IIS, Apache httpd, Nginx with AJP/HTTP proxy).
  • Capture all CFAdmin settings: datasources, mail servers, caching, Security, Scheduled tasks, Custom tags/mappings, PDFg, cfdocument, Solr, REST, SOAP services.
  • Note libraries and extensions: JDBC drivers, BouncyCastle, ImageMagick/GhostScript, FOP, POI, BC FIPS, Redis, Memcached, Ehcache, S3 SDK, Elasticsearch/Solr client versions.
  • Dependencies on external systems: Payment gateways, SSO/LDAP/ADFS/SAML/OAuth, SMTP, file shares, Message queues, internal APIs, firewall rules, SSL/TLS certs and keystores.
  • Licensing requirements or changes (ACF Enterprise -> Standard Features, or Lucee open‑source differences).

• Backups and reproducibility

  • Full backup of code, CFIDE/CFAdmin exports, server.xml/web.xml, connector configs, JVM options, and OS service definitions.

  • Database backups and anonymized datasets for testing.

  • Export CFAdmin config for Automation using CFConfig.

    • Example (CommandBox):

      box install CommandBox-cfconfig
      cfconfig export from=/path/to/cfusion to=./cfadmin.json

  • Store all of the above in Version control or a secure artifact repository.

• Version planning and compatibility

  • Identify your target runtime(s): Adobe ColdFusion 2023/2021, Lucee 5.4/6.x, Tomcat version, JDK 11/17.
  • Read official “What’s New” and “Breaking Changes” for your target version(s):
    • Secure Profile, stricter EL and script protection, null support, changes to cfimage/CFPdf behavior, cfthread nuances, default Security headers, AJP secrets, SameSite cookie settings.
  • Decide on a staged upgrade path where needed (e.g., big jumps may warrant interim versions or a dedicated compatibility pass).

• Environment parity

  • Define a test environment that mirrors production: same OS family, JDK, web server, connectors, time zones, memory limits, CPU, and network topology.

  • If containerized, use Docker images with pinned tags. Example:

    docker-compose.yml (example for Adobe CF 2023 and Lucee side-by-side)

    services:
    acf2023:
    image: adobecoldfusion/server:2023.0.07
    environment:

    • acceptEULA=YES
    • password=Admin@123
      ports: [“8500:8500”]
      lucee54:
      image: lucee/lucee:5.4-nginx-tomcat
      ports: [“8888:8888”]

  • Use distinct DNS or hostnames for each environment to avoid cross‑talk.

• Test data, credentials, and fixtures

  • Prepare anonymized, representative datasets.
  • Generate or import API keys, certificates, and secrets for non‑production.
  • Seed user accounts and roles needed for end‑to‑end flows.

• Tooling and observability

  • Install testing frameworks: TestBox (unit/Integration), Selenium or Playwright (UI), JMeter/k6 (load/stress), OWASP ZAP (security), SonarQube/Fortify (SAST).
  • Observability: FusionReactor/SeeFusion, New Relic, ELK/EFK, OpenTelemetry, System metrics dashboards.

• Change control and rollback

  • Freeze date for code changes.
  • Define rollback plan, including Database schema rollbacks and DNS reversions.
  • Pre‑approve Maintenance windows and communication templates.

---

What to test before migration: scope and strategy

Testing before migration means establishing a repeatable baseline and validating that the application behaves identically—or better—on the target platform.

Key goals

• Establish a Performance and functional baseline on the current environment.
• Build a production‑like test environment for the target runtime.
• Run automated and manual tests to surface compatibility issues.
• Measure and compare performance, memory usage, and error rates.
• Document gaps and create targeted fixes or Configuration changes.

Test coverage Checklist (high‑level)

• CFML language features (cfscript, closures, null support, date math, regex).
• Datasources (JDBC drivers, connection strings, transaction isolation, cfqueryparam types).
• Caching (template cache, object cache, Redis/Memcached/Ehcache).
• Security (secure profile, CFAdmin lockdown, CSP, CORS, SameSite cookies, TLS versions).
• PDF/Imaging (cfdocument/CFPdf/cfimage, headless fonts, GhostScript/ImageMagick).
• Search and indexing (Solr/Elasticsearch).
• Email and queuing (cfmail, SMTP auth/TLS, Message queues).
• Scheduled tasks and cfthread behavior.
• File system I/O, UNC paths, container volumes.
• Web server connectors and session stickiness (IIS/Apache/AJP).
• REST/SOAP endpoints, JSON/XML serialization differences.
• Logging/monitoring integrations and alerting.

---

Step‑by‑Step Pre‑Migration Testing and Dry‑Run Migration

1. Create a baseline in your current environment

• Enable extra logging around errors, deprecated features, and Slow queries.
• Run a full regression suite (unit, Integration, and UI) and capture:
  • Peak RPS, average/95th latency, CPU/memory footprint.
  • Error/exception counts and top stack traces.
• Save a “golden” dataset snapshot for consistent re‑runs.

2. Export Configuration from CFAdmin

• Use CFConfig to export datasources, mail servers, mappings, cache, and Security settings.

  cfconfig export to=./cfadmin.json

• Keep a copy of web server configs:

  • IIS: applicationHost.config, BonCode/ISAPI filters, wsconfig settings.
  • Apache: httpd.conf, mod_jk/mod_proxy_ajp/proxy configs.
  • Tomcat: server.xml (HTTP/AJP connectors), web.xml, context.xml.

3. Stand up a target environment

• On bare metal/VM:

  • Install the target CF engine, Tomcat, and JDK (pin versions).
  • Apply the latest cumulative updates/hotfixes.

• In containers:

  • Use pinned Docker tags and reproducible compose files.

• Start ColdFusion under CommandBox for rapid Iteration:

  box server start cfengine=adobe@2023.0.07+330123 jvmargs=”-Xms1g -Xmx2g -Dfile.encoding=UTF-8″
  box server start cfengine=lucee@5.4.4 jvmargs=”-Xms1g -Xmx2g”

4. Import configuration and adapt to the new runtime

• Apply your CFAdmin export:

  cfconfig import from=./cfadmin.json

• Verify and adjust:

  • Datasource JDBC URLs and drivers (e.g., switch from jTDS to Microsoft JDBC for SQL Server; ensure TLS 1.2+).
  • Mail server TLS/auth settings.
  • Caching and Session replication configs.
  • PDFg service endpoints and fonts.
  • Security profile, SameSite cookies, secure and httpOnly flags.

5. Configure web server connectors and app Server security

• For Adobe CF: use wsconfig to connect IIS/Apache, or BonCode for IIS AJP.

• Ensure AJP is secured:

  • Tomcat server.xml example:

• Confirm HTTPS offload, HSTS, and TLS cipher suites.

6. Smoke test Deployment and basic health checks

• Deploy your code to the target environment.

• Add a lightweight health endpoint:

  
  writeOutput( serializeJSON({
  status=”ok”,
  version=server.coldfusion.productVersion,
  jvm=server.java.version,
  time=now()
  }) );
  

• Validate 200 responses, session initialization, and static asset delivery.

7. Run automated test suites

• Unit/integration with TestBox:

  component {
  function run( testResults ){
  describe(“User Service”, function(){
  it(“creates a user”, function(){
  var svc = new path.to.UserService();
  var user = svc.create(name=”Alice”);
  expect( user.id ).toBeTypeOf(“numeric”);
  });
  });
  }
  }

• UI flows with Selenium/Playwright targeting the new base URL.

• Compare pass/fail between current and target environments.

8. Validate datasources and ORM/Hibernate behavior

• Programmatically confirm datasource connectivity:

  
  for(ds in server.datasources){
  try{
  q = queryExecute(“SELECT 1”, [], {datasource=ds});
  writeOutput(ds & “: OK
  “);
  } catch(any e){
  writeOutput(ds & “: FAIL – ” & e.message & “
  “);
  }
  }
  

• Check for:

  • Strict type mismatches with cfqueryparam (cfsqltype).
  • Changes in date/time handling and time zones.
  • Hibernate upgrades (lazy loading, proxies, dialects, batch sizes).

9. Exercise “edges and extras”

• PDF/Imaging: CFDocument, CFPdf, CFImage, fonts, page margins, rasterization, barcode generation.
• Search: Solr core schema, index rebuilds, analyzers, stopwords.
• Scheduler: confirm tasks, cron expressions, credentials, and file paths.
• CFThread and asynchronous code: thread pools, timeouts, exception handling.
• Serialization: JSON/XML formats, null handling, date formatting.

10. Performance and load testing

• Replay representative traffic using JMeter/k6/Gatling.

• Record:

  • Throughput, latency, error rates, GC pauses, heap usage.
  • DB CPU/IO, slow SQL, connection pool saturation.

• Tune JVM and ColdFusion settings:

  • Example jvm.config:

    -Xms2048m
    -Xmx4096m
    -XX:+UseG1GC
    -Duser.timezone=UTC
    -Dfile.encoding=UTF-8

  • Adjust CF request timeouts, pool sizes, and trusted cache.

11. Security and Compliance checks

• Run OWASP ZAP/DAST against the target environment.
• Confirm CF Administrator lockdown or secure profile:
  • RDS disabled.
  • CFAdmin behind IP allowlist/VPN or removed from prod.
  • Session cookies: Secure, HttpOnly, SameSite=Lax/Strict as policy dictates.
• Validate TLS versions, keystores, and certificate chains.

12. Document issues, fix, and iterate

• Track each deviation between old and new runtime.
• Categorize and resolve: code changes, configuration changes, or dependency updates.
• Re‑run the same test suite to confirm resolution; keep everything scripted to enable repeatability.

---

Practical examples and configuration snippets

Example: strict cfqueryparam types

If the target runtime is stricter about numeric types, update queries:

SELECT * FROM orders WHERE id =

Example: date/time parsing differences

Avoid locale‑dependent parsing:

Example: enabling null support

• In Adobe CF 2021/2023, review null support and serialization settings in CFAdmin.
• In Lucee, enable null support where required:

this.nullSupport = true; // in Application.cfc

Example: logging configuration

Centralize logs and add request IDs:

log.logEntry( type=”information”, text=”Order placed”, application=”orders”, logpath=”/var/log/coldfusion/app.log” );

---

Risks, Common Issues, and How to Avoid Them

• JDBC and TLS changes

  • Risk: Old jTDS driver or outdated JDBC causing SSL handshake failures.
  • Avoid: Use vendor-supported drivers (Microsoft/Oracle/MariaDB), enable TLS 1.2+, update connection strings with truststore paths if needed.

• Secure profile and defaults

  • Risk: CFAdmin secure profile breaks legacy features (RDS, debug output, directory browsing).
  • Avoid: Explicitly configure needed features, secure CFAdmin behind VPN, and never expose CFIDE to the internet.

• AJP connector restrictions

  • Risk: Tomcat 9+ requires AJP secrets; misconfig leads to 503s.
  • Avoid: Configure secretRequired with a strong secret and lock AJP to localhost.

• Session and cookie behavior

  • Risk: SameSite=Lax/Strict blocking cross‑site auth flows, changes to secure/httpOnly defaults.

  • Avoid: Set cookies deliberately in Application.cfc:

    this.sessionCookie = { secure=true, httpOnly=true, sameSite=”Lax” };

• PDF/Imaging differences

  • Risk: Font substitution, layout shifts, or 32‑bit toolchain dependencies.
  • Avoid: Package fonts, test on headless servers, and explicitly set fonts in CFDocument.

• Null and serialization

  • Risk: Null support alters JSON payloads; downstream clients break.
  • Avoid: Standardize JSON serialization settings and contract tests for APIs.

• ORM/Hibernate regressions

  • Risk: Upgraded Hibernate changes lazy loading, query cache, or dialects.
  • Avoid: Validate entity mappings, second‑level cache config, and batch fetch sizes.

• Time zone and locale

  • Risk: Different defaults cause date math drift and test flakiness.
  • Avoid: Force UTC in JVM and app; format/parse using ISO 8601.

• File paths and permissions

  • Risk: UNC paths, container volumes, or SELinux/AppArmor differences.
  • Avoid: Validate paths, mount volumes explicitly, and set correct service accounts.

• Feature parity when moving ACF → Lucee

  • Risk: Tag/function differences, Admin settings nomenclature, extension differences.
  • Avoid: Use the Lucee Compatibility guide, test CFDocument/CFPdf alternatives, and isolate engine‑specific code paths with feature flags.

---

Post‑Migration Checklist and Validation Steps

• Application readiness

  • All automated test suites pass against the target environment.
  • Manual exploratory tests completed for critical user journeys.
  • No P1/P2 defects open; all P3+ risks documented and accepted.

• Performance and stability

  • Throughput and latency equal to or better than baseline.
  • GC behavior stable; no sustained Memory leaks or full‑GC thrashing.
  • Database metrics within acceptable thresholds; Slow queries optimized.

• Security and Compliance

  • Secure profile/lockdown applied; CFIDE/CFAdmin access controlled.
  • TLS configuration validated; certificates current and pinned.
  • Vulnerability scans (SAST/DAST) reviewed and critical items remediated.

• Configuration integrity

  • Datasources tested, failover verified where applicable.
  • Mail server Authentication and TLS confirmed with test emails.
  • Scheduler tasks running on the correct node(s) with proper credentials.
  • Cache backends (Redis/Memcached/Ehcache) connected and sized properly.

• Integrations and services

  • Payment, SSO/LDAP/SAML/OAuth, internal APIs tested with contract tests.
  • Search indices rebuilt and validated; query relevance tested.
  • PDF/Imaging output visually compared; fonts embedded where required.

• Web tier and routing

  • IIS/Apache connectors healthy; AJP secrets configured; sticky sessions verified in load balancers.
  • Health checks and readiness endpoints registered with the load balancer.
  • Error pages and logging work as expected in non‑debug mode.

• Observability and operations

  • FusionReactor/New Relic agents running; dashboards updated for new hosts.
  • Centralized logging aggregated; alerts tuned to new baselines.
  • Backups and runbooks updated; on‑call trained on new procedures.

• Rollback and cutover

  • Rollback steps rehearsed; database migrations reversible.
  • DNS/traffic switchover plan defined with timed checkpoints.
  • Stakeholders sign off; Maintenance window approved.

Example go‑live smoke script:

• Hit /health and /version endpoints.
• Log in, create a user, place a test order, generate a PDF invoice, send an email, and verify DB rows and logs.
• Run a 15‑minute light load test to validate stability before opening full traffic.

---

Reference table: areas to test and tools

Area	What to test	Tools/Notes
Language/CFML	Nulls, closures, date math, serialization	TestBox, contract tests
Datasources/JDBC	TLS, pooling, types, transaction isolation	Vendor JDBC, queryExecute checks
Caching/Sessions	Template cache, Redis/Memcached, stickiness	Load balancer, cache hit ratio monitoring
PDF/Imaging	Fonts, layout, performance	Visual diffs, embedded fonts
Search	Index rebuilds, analyzers, synonyms	Solr admin, relevance tests
Scheduler/Threads	Cron correctness, concurrency, Error handling	Logs, admin screens, idempotent task design
Security	Lockdown, cookies, headers, TLS	ZAP, headers analyzer, CFAdmin
Web Connectors	AJP/HTTP proxy, secrets, timeouts	Tomcat server.xml, IIS/Apache configs
Observability	Metrics, traces, logs, alerts	FusionReactor/New Relic/ELK
Performance	Throughput, latency, GC, DB	JMeter/k6, profiler, DB monitor

---

Governance: definitions of done and sign‑off

• Functional parity documented and confirmed.
• Performance not worse than baseline by agreed thresholds.
• Security posture meets policy (CIS/OWASP/PCI/etc.).
• Operations runbooks, dashboards, and alerts updated.
• Rollback tested and feasible within the maintenance window.

---

FAQ

How far back can I upgrade directly without intermediate steps?

Large version jumps can introduce multiple breaking changes at once. While you can often move directly to Adobe ColdFusion 2023, consider an intermediate validation on 2021 if your codebase relies on features deprecated in 2018 or earlier. For Lucee, verify tag/function parity and extension availability before attempting a single-step leap.

Should I switch from Adobe ColdFusion to Lucee during the same upgrade?

Combining engine migration with an upgrade multiplies risk. Many teams first upgrade within Adobe ColdFusion to get current and then plan a separate ACF → Lucee migration. If timelines force a single move, isolate engine‑specific areas behind feature flags and maintain a robust automated test suite.

How do I test scheduled tasks and CFThread behavior safely?

Run scheduled tasks against non‑production endpoints and data; add “dry run” switches to avoid side effects. For CFThread, log thread names and correlation IDs, set timeouts, and verify exception handling. Test under concurrent load to surface pool starvation or deadlocks.

What’s the best way to handle production data in tests?

Prefer anonymized datasets that preserve shape, ranges, and edge cases. For integrations requiring real data (e.g., SSO assertions), set up test tenants or sandboxes. Never move raw PII to non‑production without proper masking and approvals.

How long should my soak test be before cutover?

At minimum, run a 1–2 hour steady‑state test to observe memory/GC patterns and slow creep issues. For critical systems, an overnight (6–12 hour) soak gives higher confidence, especially when scheduled tasks and maintenance jobs execute.