wpdev/2meet-data-optimizer
1
0
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases 1 Wiki Activity

hardening 2.3: 2meet-data-optimizer 1.3.0 (flat-table DDL outside web requests, routed count cache, audit fixes) #7

Merged
wpdev merged 20 commits from hardening/2.3 into main 2026-09-27 18:19:18 +08:00
Conversation 0 Commits 20 Files Changed 30
+1193 -105
wpdev commented 2026-09-27 18:03:29 +08:00
Owner
Copy Link
Copy Source

Release 2.3 (1.3.0): flat-table DDL outside web requests, routed count cache, audit fixes.

Commits:

  • docs(2meet-data-optimizer): add PR-116 and PR-117 to the 1.3.0 release notes
  • perf(2meet-data-optimizer): keep cached routed counts across unrelated post meta writes
  • perf(2meet-data-optimizer): key the routed count cache on the SQL without wpdb's placeholder escape
  • docs(2meet-data-optimizer): add PR-111, PR-113 and PR-114 to the 1.3.0 release notes
  • fix(2meet-data-optimizer): close the gaps in deferred table upgrades found in the 2.3 review
  • fix(2meet-data-optimizer): keep the audit trail of mode changes and snapshot deletions
  • fix(2meet-data-optimizer): alter an existing flat table in one process at a time; keep a deferred group native until its drift fix
  • docs(2meet-data-optimizer): add PR-106 to PR-110 to the 1.3.0 release notes
  • perf(2meet-data-optimizer): flush the cold cache group without a server-side scan
  • fix(2meet-data-optimizer): take the index lock on the primary under read/write splitting
  • fix(2meet-data-optimizer): upgrade existing flat tables from WP-CLI or WP-Cron only
  • fix(2meet-data-optimizer): forget post status columns of groups whose flat table is gone
  • docs(2meet-data-optimizer): 1.3.0 release notes
  • chore(2meet-data-optimizer): prefix the global variables in uninstall.php
  • refactor(2meet-data-optimizer): handle admin form posts before the page renders
  • fix(2meet-data-optimizer): treat contact and address keys as private in the public REST filter
  • fix(2meet-data-optimizer): log info and notice events only when debug logging is on
  • perf(2meet-data-optimizer): stop writing an option for every rate-limited view
  • perf(2meet-data-optimizer): count routed searches from a cached base count minus the excluded posts
  • fix(2meet-data-optimizer): add declared flat-table indexes from WP-CLI or WP-Cron only, one process at a time

Acceptance: notes/tmdo-acceptance-2.3-2026-09-27.md (dev36, 902k listings).

🤖 Generated with Claude Code

Release 2.3 (1.3.0): flat-table DDL outside web requests, routed count cache, audit fixes. Commits: - docs(2meet-data-optimizer): add PR-116 and PR-117 to the 1.3.0 release notes - perf(2meet-data-optimizer): keep cached routed counts across unrelated post meta writes - perf(2meet-data-optimizer): key the routed count cache on the SQL without wpdb's placeholder escape - docs(2meet-data-optimizer): add PR-111, PR-113 and PR-114 to the 1.3.0 release notes - fix(2meet-data-optimizer): close the gaps in deferred table upgrades found in the 2.3 review - fix(2meet-data-optimizer): keep the audit trail of mode changes and snapshot deletions - fix(2meet-data-optimizer): alter an existing flat table in one process at a time; keep a deferred group native until its drift fix - docs(2meet-data-optimizer): add PR-106 to PR-110 to the 1.3.0 release notes - perf(2meet-data-optimizer): flush the cold cache group without a server-side scan - fix(2meet-data-optimizer): take the index lock on the primary under read/write splitting - fix(2meet-data-optimizer): upgrade existing flat tables from WP-CLI or WP-Cron only - fix(2meet-data-optimizer): forget post status columns of groups whose flat table is gone - docs(2meet-data-optimizer): 1.3.0 release notes - chore(2meet-data-optimizer): prefix the global variables in uninstall.php - refactor(2meet-data-optimizer): handle admin form posts before the page renders - fix(2meet-data-optimizer): treat contact and address keys as private in the public REST filter - fix(2meet-data-optimizer): log info and notice events only when debug logging is on - perf(2meet-data-optimizer): stop writing an option for every rate-limited view - perf(2meet-data-optimizer): count routed searches from a cached base count minus the excluded posts - fix(2meet-data-optimizer): add declared flat-table indexes from WP-CLI or WP-Cron only, one process at a time Acceptance: notes/tmdo-acceptance-2.3-2026-09-27.md (dev36, 902k listings). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
wpdev added 20 commits 2026-09-27 18:03:29 +08:00
fix(2meet-data-optimizer): add declared flat-table indexes from WP-CLI or WP-Cron only, one process at a time ec8d69a6f4 
- ensure_group_indexes() runs the ALTER only under WP-CLI or WP-Cron (filter
  tmdo_schema_ddl_allowed); web requests, including /wp-admin/ before
  auth_redirect() and admin-post.php, schedule tmdo_ensure_group_indexes instead
- MySQL GET_LOCK keeps it to one process (add_option() is INSERT ... ON
  DUPLICATE KEY UPDATE and cannot serve as a lock); released by MySQL if the
  process dies
- the applied signature is recorded only when every ALTER succeeds; failures
  log schema.index_failed and stay in pending_indexes(), which doctor reports
- new tables get their declared cidx_ indexes in CREATE TABLE
- test stub wpdb::query()/insert() return false on SQL errors like wpdb
- Verify: GroupCompositeIndexTest 7/7, unit 698, integration 576, phpcs, stan
- PR: PR-090a
perf(2meet-data-optimizer): count routed searches from a cached base count minus the excluded posts b187c1c4d3 
- a routed search with post__not_in (HivePress sends a random set per page)
  caches the COUNT without the exclusion and subtracts a live primary-key
  count of the excluded posts that match; the total stays exact
- the excluded count drops STRAIGHT_JOIN so MySQL starts from posts.ID
- Verify: PostQueryRouterFoundRowsTest, unit 698, integration 577, phpcs, stan;
  dev36 live router suite and keyword + price search
- PR: PR-091
perf(2meet-data-optimizer): stop writing an option for every rate-limited view 22b46ad175 
- blocked POST /listings/{id}/view requests update wpdo_rl_stats 1 in 10,
  each sample adding 10: anonymous traffic no longer turns 1:1 into DB writes,
  and the admin top list stays representative (unbiased estimate)
- Verify: RestApiTest counts option writes (about 20 for 200 blocked requests),
  unit 698, integration 577, phpcs, stan
- PR: PR-094
fix(2meet-data-optimizer): log info and notice events only when debug logging is on 1bdd51d9a3 
- TMDO_Logger::info()/notice() write to the PHP error log only with
  WP_DEBUG_LOG (filter tmdo_log_info); every CLI run and production request
  printed [WPDO][INFO] lines
- debug() keeps its WP_DEBUG gate; warning() and error() are unchanged
- Verify: LoggerTest (2 new), unit 700, integration 577, phpcs, stan
- PR: PR-095
fix(2meet-data-optimizer): treat contact and address keys as private in the public REST filter a268470b60 
- rest_exposable() also withholds keys named like an email, phone, address or
  IP, even when a field declares show_in_rest (defence in depth)
- Verify: RestApiTest (private and look-alike public keys), unit 701,
  integration 577, phpcs, stan; dev36 live rest suite
- PR: PR-096
refactor(2meet-data-optimizer): handle admin form posts before the page renders 1b50df7cd0 
- the main page's form handlers move unchanged from render_page() to
  handle_post(), hooked on load-{page}: inside render_page() the admin
  header was already sent, so their wp_safe_redirect() calls could not work
- the setup wizard's step 4 snapshot post moves to handle_snapshot_post() on
  its load hook; a failure redirects with snap_failed=1 for the notice
- Verify: unit 701, integration 577, phpcs, stan; dev36 load hook handles a
  posted reset and redirects before output
- PR: PR-097
chore(2meet-data-optimizer): prefix the global variables in uninstall.php d46913de41 
- $bootstrap_files, $bootstrap_ok, $f, $result, $snapshot_id and $dir become
  $wpdo_* like the file's other globals (WordPress.org Plugin Check)
- Verify: PrefixAllGlobals sniff 0, phpcs, UninstallGuardTest and
  InstallerCleanupTest, unit 701
- PR: PR-098
docs(2meet-data-optimizer): 1.3.0 release notes 303781e68b 
- version 1.3.0; CHANGELOG and readme.txt for PR-090a, 091, 093 (verified,
  no change), 094–098, with the post-upgrade steps (first CLI / cron run adds
  the HivePress indexes; INFO lines need WP_DEBUG_LOG)
- Verify: composer test (701)
- PR: PR-101
fix(2meet-data-optimizer): forget post status columns of groups whose flat table is gone 5968bcd95d 
- on_transition() skips a recorded group whose table no longer exists and
  removes it from tmdo_post_status_columns; every post status change used to
  UPDATE the missing table and print a database error (seen on dev36 after a
  throwaway group's table was dropped)
- Verify: PostStatusSyncTest (dropped table), integration 578, unit 701, phpcs, stan
- PR: PR-106
fix(2meet-data-optimizer): upgrade existing flat tables from WP-CLI or WP-Cron only e18030b206 
- a changed group definition no longer runs dbDelta on an existing table
  inside a web request (seconds to minutes on a large table, and every
  concurrent request tried it): the group is recorded in tmdo_schema_deferred
  and tmdo_ensure_group_indexes is scheduled, whose request upgrades it
- meanwhile the group stays native: Hook Bus field lookups return no field and
  the router treats it as drifting; afterwards a drift fix
  (tmdo_drift_resync) rebuilds the writes that skipped the flat table
- new tables are still created at once; aeav_only upgrades at once (the flat
  table is the only copy); doctor lists deferred groups
- Verify: SchemaUpgradeDeferralTest, Hook Bus and router tests, integration
  582, unit 701, phpcs, stan; dev36 live cycle in the 2.3 acceptance
- PR: PR-107
fix(2meet-data-optimizer): take the index lock on the primary under read/write splitting 06a3594417 
- when $wpdb has send_reads_to_masters() (HyperDB, LudicrousDB), it is called
  before SELECT GET_LOCK, which a replica would take without locking out the
  ALTER on the primary
- Verify: GroupCompositeIndexTest (call order through a wpdb proxy),
  integration 583, unit 701, phpcs, stan
- PR: PR-108
perf(2meet-data-optimizer): flush the cold cache group without a server-side scan eb1c8b78f4 
- TMDO_Cache_Layer::flush_group() bumps the post type's cold cache version
  (option wpdo_cold_cache_version) instead of wp_cache_flush_group(), which
  Redis Object Cache runs as one server-side scan that blocks Redis on a large
  group; entries under older keys expire by their TTL
- TMDO_Zone_Cold::cache_key() builds every cold blob key; version 0 keeps the
  current keys, so upgrading invalidates nothing
- Verify: CacheLayerTest (new keys, no scan, other post types untouched),
  unit 702, integration 583, phpcs, stan
- PR: PR-110
docs(2meet-data-optimizer): add PR-106 to PR-110 to the 1.3.0 release notes 0aec4b9bb6 
- Changed: deferred table upgrades (PR-107), index lock on the primary
  (PR-108), cold cache flush by version (PR-110); Fixed: dropped tables in
  post status sync (PR-106)
- Verify: composer test
- PR: PR-101 (follow-up)
fix(2meet-data-optimizer): alter an existing flat table in one process at a time; keep a deferred group native until its drift fix 29adb05567 
- security review H1: anonymous /wp-cron.php requests pass ddl_allowed(),
  and dbDelta on an existing table had no lock, so concurrent requests each
  ran the ALTER on a large table. The upgrade now runs under the same MySQL
  GET_LOCK as the index DDL (skipped when another process holds it) and
  re-checks the stored hash once locked; this covers aeav_only too. The table
  build moves unchanged to write_table(); lock()/unlock() are shared
- security review M1: after a deferred upgrade the group stays native until
  tmdo_drift_resync has rebuilt the rows written meanwhile
  (TMDO_Schema_Manager::resynced() from run_scheduled_fix()); with no flat
  writes (disabled) it is released at once
- Verify: SchemaUpgradeDeferralTest (lock held elsewhere, resync release),
  integration 585, unit 702, phpcs, stan
- PR: PR-111 (2.3 acceptance security review)
fix(2meet-data-optimizer): keep the audit trail of mode changes and snapshot deletions e26aadc4ec 
- TMDO_Logger::notice() is always written to the error log (it had no
  callers); bridge_mode_changed and snapshot_deleted use it. Since PR-095
  info() is silent without WP_DEBUG_LOG, which left no record of who switched
  a mode or deleted a snapshot (security review L1)
- Verify: LoggerTest (notice always logged, info gated), unit 703,
  integration 585, phpcs, stan
- PR: PR-113 (2.3 acceptance security review)
fix(2meet-data-optimizer): close the gaps in deferred table upgrades found in the 2.3 review 1d70d13f45 
- a deferred group whose stored definition matches again (reverted, or
  upgraded by another process) goes through end_deferral() instead of staying
  native for good (review F2)
- the tmdo_ensure_group_indexes handler upgrades deferred tables itself: with
  ALTERNATE_WP_CRON the event runs after init in a visitor's request (F3)
- a new group table created while the entity writes to flat stays native
  until the drift fix has filled it from native meta; an opted-in HivePress
  attribute no longer routes searches to an empty table (F6)
- direct_read() returns native values for a deferred group (F10); the Hook
  Bus ignores deferral in aeav_only, as the router does
- dbDelta failures: the definition is recorded only when every column exists
  (schema.upgrade_failed otherwise), so a failed ALTER is retried and the
  group is not released onto missing columns
- test bootstrap: get_metadata() reads the real meta table
- Verify: SchemaUpgradeDeferralTest (6), Hook Bus direct read, integration
  587, unit 703, phpcs, stan
- PR: PR-114 (2.3 acceptance correctness review)
docs(2meet-data-optimizer): add PR-111, PR-113 and PR-114 to the 1.3.0 release notes 1835c8776a 
- deferred-upgrade hardening from the acceptance reviews, notice() audit
  trail, known cold-cache version window
- PR: PR-101 (follow-up)
perf(2meet-data-optimizer): key the routed count cache on the SQL without wpdb's placeholder escape 7851f150fa 
- keyword searches carry wpdb's placeholder escape for "%" ({<hash>}), which
  is new in every request, so md5(count SQL) never matched across requests
  and every keyword search recounted (the 2.2 "keyword + filter routed slower
  than native" finding). The key now uses remove_placeholder_escape()
- found in the 2.3 acceptance: dev36 keyword + price routed 6.2 s vs native
  4.9 s; PR-091's in-process check reused one process, one escape
- Verify: PostQueryRouterFoundRowsTest (two escapes, one COUNT), integration
  588, unit 703, phpcs, stan; dev36 front-end
- PR: PR-116 (2.3 acceptance)
perf(2meet-data-optimizer): keep cached routed counts across unrelated post meta writes ca2cbdab65 
- cached counts followed posts' last_changed, which WordPress moves on any
  post meta write; themes write meta on page views (on dev36 Blocksy saved a
  dynamic-CSS descriptor for each listing a crawler visited), so the cache
  was invalidated every few seconds and keyword/filter searches recounted
- counts now follow their own last_changed group (tmdo_posts), bumped on
  clean_post_cache (post saved or deleted; hooked in every mode) and on flat
  writes through the Hook Bus, plus terms' last_changed; a routed search reads
  no unmanaged meta
- Verify: PostQueryRouterFoundRowsTest (unrelated meta write keeps the count,
  save or flat write recounts), integration 588, unit 703, phpcs, stan;
  dev36 front-end
- PR: PR-117 (2.3 acceptance)
docs(2meet-data-optimizer): add PR-116 and PR-117 to the 1.3.0 release notes
Anti-EAV Lint + Quality Gate / anti-eav-lint (pull_request) Successful in 24s
Details
Tests / PHP Lint (pull_request) Successful in 40s
Details
Tests / Unit Tests (pull_request) Successful in 41s
Details
Tests / Integration Tests (pull_request) Successful in 1m23s
Details
Tests / PHPCS (pull_request) Successful in 29s
Details
Tests / PHPStan (pull_request) Successful in 1m14s
Details
947e130386 
- routed count cache: placeholder escape and count version group
- PR: PR-101 (follow-up)
wpdev closed this pull request 2026-09-27 18:16:23 +08:00
wpdev reopened this pull request 2026-09-27 18:16:24 +08:00
wpdev merged commit f929165b7b into main 2026-09-27 18:19:18 +08:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
wpdev
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: wpdev/2meet-data-optimizer#7
Delete Branch "hardening/2.3"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?