Commit Graph

101 Commits

Author SHA1 Message Date
wpdev 6f117f4e9b fix(2meet-data-optimizer): 1.4.2 acceptance fixes for archive sweep and uninstall
Anti-EAV Lint + Quality Gate / anti-eav-lint (pull_request) Successful in 21s
Tests / PHP Lint (pull_request) Successful in 18s
Tests / Unit Tests (pull_request) Successful in 41s
Tests / Integration Tests (pull_request) Successful in 1m25s
Tests / PHPCS (pull_request) Successful in 44s
Tests / PHPStan (pull_request) Successful in 47s
- Archive sweep waits until idx_original_meta exists (a plugin update does
  not run dbDelta; without the index NOT EXISTS scans the archive per row)
- Duplicate archive cleanup runs from WP-CLI only, never in a web activation
- Uninstall keeps the LatePoint add-on's tmdo_latepoint_db_version
- AJAX save errors stay on screen instead of fading after 2.5 s
- Verify: 2 new integration tests; unit 711, integration 612, phpcs 0, stan OK
- PR: PR-141 (2.4.2 correctness review)
2026-09-28 18:43:50 +08:00
wpdev 15096f064f fix(2meet-data-optimizer): deactivate and uninstall clear every scheduled event and tmdo_* option
- One CRON_HOOKS list for deactivate() and uninstall; wp_unschedule_hook() also
  removes events scheduled with args (tmdo_drift_resync, backfill batches).
- Fix the wrong wpdo_user_stress_test_batch name; add migration tick,
  entity backfill, auto-promote and all stress-test batch hooks.
- Uninstall sweeps tmdo_* options and _transient_(timeout_)tmdo_* rows too.
- Verify: InstallerCleanupTest (3 new), unit 711, integration 611, phpcs 0
- PR: PR-138 (manual F-4, F-5)
2026-09-28 18:25:01 +08:00
wpdev 90877b81e0 fix(2meet-data-optimizer): entity health sees a running backfill
- Backfill batches are scheduled with (entity, group) args; the health
  panel looked for (entity) alone, never saw one and kept offering to start
  a backfill, which resets the group's checkpoint
- is_backfill_active() checks every group of the entity
- Unit bootstrap: wp_next_scheduled() stub honours hook + args
- Verify: EntityHealthBackfillTest (2 new); unit 711, phpcs 0, stan OK
- PR: PR-137 (manual F-11)
2026-09-28 18:18:56 +08:00
wpdev 6687d54bef fix(2meet-data-optimizer): archive sweep copies each trashed meta row once
- sweep() skips meta rows already in wpdo_archive (NOT EXISTS on
  original_meta_id): it copied the same trashed meta again every day and
  wpdo_archive has no unique key (dev20: 300 rows for 150 meta rows)
- TMDO_Zone_Archive::remove_duplicates() keeps the first copy per
  original_meta_id (rows without one are left alone); install() runs it,
  so `wp tmdo install` cleans existing sites
- wpdo_archive gets KEY idx_original_meta for the lookup (new installs;
  existing sites via `wp tmdo install`). SCHEMA_VERSION is not bumped, so
  no web request runs dbDelta
- Verify: ZoneArchiveIntegrationTest (2 new); dev20 read-only check: 150 of
  300 rows would be removed; unit 709, integration 608, phpcs 0, stan OK
- PR: PR-136 (manual F-6)
2026-09-28 18:17:47 +08:00
wpdev 86ac84e07c fix(2meet-data-optimizer): show why an admin bridge-mode change was refused
- The settings form and the AJAX section save both went through
  TMDO_Mode_Manager::set() and dropped its WP_Error: an unsafe jump or a
  Promotion Guard refusal still said "saved" while the mode stayed put
- TMDO_Admin::apply_bridge_modes() returns the refusals; the full form keeps
  them in a per-user transient and shows them after the redirect, the AJAX
  save returns them as the error message (the settings JS shows it)
- Verify: AdminBridgeModeSaveTest (5 new); unit 709, integration 606, phpcs 0, stan OK
- PR: PR-135 (manual F-2)
2026-09-28 18:14:35 +08:00
wpdev 3e96357459 fix(2meet-data-optimizer): give new posts' flat rows their status/date copies; sync drops cached counts
- add_flat_rows (wp_insert_post, after transition_post_status) inserts the
  copies with the empty row: a NULL copy turned the copy route off for the
  whole group until the next sync
- wp tmdo post-status-sync bumps the routed count version (not on dry runs)
- Test bootstrap: minimal WP_Post stub
- Found in the 2.4 correctness review (residual risks)
- Verify: 2 new integration tests; integration 606, unit 704, phpcs 0, stan OK
- PR: PR-133
2026-09-28 00:32:59 +08:00
wpdev 80343ec282 fix(2meet-data-optimizer): drop router query vars a caller passed in
- pre_get_posts clears wpdo_post_clauses / _sort / _copy_route / _count_sql
  before anything else: new WP_Query( array( … ) ) could preset them
- posts_orderby whitelists the copy route's direction
- Found in the 2.4 security review (Low L2; not reachable from URLs)
- Verify: PostQueryRouterCopyTest (1 new); integration 604, unit 704, phpcs 0
- PR: PR-132
2026-09-28 00:31:10 +08:00
wpdev 33addd1e49 fix(2meet-data-optimizer): rewrite only standalone posts.ID tie-breaks in ORDER BY
- flat_first and the copy route share ties_on_post_id(): a posts.ID term
  alone (optionally with a direction) reads the flat post_id; posts.ID inside
  an expression a plugin added (FIELD(), IN) is left alone — inserting a
  direction there made the SQL invalid and the page empty
- PCRE failures return the text given instead of '' (the query kept its
  shape rather than becoming an empty request)
- Found in the 2.4 correctness review (Medium) and security review (Low L1)
- Verify: 2 new integration tests (the flat-first one runs the SQL), router suites 69 OK
- PR: PR-131
2026-09-28 00:29:53 +08:00
wpdev bed3f906da fix(2meet-data-optimizer): keep meta_key native when the query still orders by it
- meta_key is routed as EXISTS only when no orderby entry names meta_value,
  meta_value_num or the key (several orderings or an array included);
  clearing it made WordPress drop the price ordering
- A sort group is dropped as unused only when orderby names neither it,
  meta_value(_num) nor its key (it is then the primary clause)
- meta_compare_key other than '=' keeps meta_key native (LIKE matches other keys)
- Found in the 2.4 correctness review (regressions from PR-122)
- Verify: PostQueryRouterTest (3 new), router suites 67 OK
- PR: PR-130
2026-09-28 00:27:56 +08:00
wpdev 692b23aed9 fix(2meet-data-optimizer): drop cached routed counts when the migration engine rebuilds post rows
- migrate_entity (drift-check --fix), migrate_group and migrate_group_batch
  write flat rows outside the Hook Bus: bump the routed count version
  (once per call / batch; not on dry runs; post rows only)
- dev36: a vendor search kept counting 21515 of 21522 matches after a fix
- Verify: composer test (704), test:integration (598), phpcs, stan
- PR: PR-129
2026-09-27 23:25:03 +08:00
wpdev 8ee3dce0fe perf(2meet-data-optimizer): let the status/date copy route stop at the LIMIT
- Copy route: drop WordPress's GROUP BY when the flat row is the only join,
  and read posts.ID tie-breaks as the flat post_id (own direction, once)
- Post status sync: index (post_status, post_date, post_id) replaces the
  two-column idx_post_status_date (added / dropped by post-status-sync)
- dev36, 235k of 1.3M listings in a price range: main query 1.7 s -> 3 ms
- Verify: composer test (704), test:integration (597), phpcs, stan
- PR: PR-128
2026-09-27 23:12:34 +08:00
wpdev 2aa9058ee9 fix(2meet-data-optimizer): keep a post row's status and date copies when the migration engine rebuilds it
- migrate_entity() (drift-check --fix, per-row backfill) REPLACEs the whole
  flat row and left out the post_status / post_date copies, so a rebuilt row
  had NULL status. One such row turns the status/date copy route off for the
  whole group: on dev36 a single row rebuilt by the stage-3 drift fix made the
  date-ordered price search 1.47 s instead of driving from the flat index
- post rows now carry TMDO_Post_Status_Sync::insert_columns(); the bulk pivot
  backfill (ON DUPLICATE KEY UPDATE) was not affected
- Verify: PostStatusSyncTest (existing and new rows keep their copies),
  integration 593, unit 704, phpcs, stan; dev36 post-status-sync + front-end
- PR: PR-127 (found in 2.4 profiling)
2026-09-27 22:58:09 +08:00
wpdev 66db9e4c56 perf(2meet-data-optimizer): route HivePress's featured-listing lookup
- HivePress runs it on every listing search page from its own
  pre_get_posts: get_posts() with the search's meta query (still unrouted,
  including its sort group), meta_key hp_featured and orderby rand. The
  router declined both shapes, so the lookup stayed native: 2.26–2.38 s of a
  2.5 s page at 1.3M listings on dev36
- a sort group (key NOT EXISTS OR key EXISTS) that no orderby uses matches
  every post and is dropped; meta_key without meta_value and without a
  meta_value sort routes as EXISTS (WP_Query turns it into that clause)
- Verify: PostQueryRouterTest (featured lookup routed, meta_key with a value
  untouched), integration 592, unit 704, phpcs, stan; dev36 front-end
- PR: PR-122 (follow-up)
2026-09-27 22:54:17 +08:00
wpdev 2ed48c3ba3 perf(2meet-data-optimizer): route meta clauses without a value as EXISTS
- WP_Meta_Query adds a value condition only when 'value' is set, so a clause
  like array( 'key' => 'hp_featured' ) only tests that the key exists. The
  router read it as "= ''", declined it, and kept the whole query native
- HivePress runs such a query on every listing search page (featured-listing
  lookup in Attribute::set_search_query(): hp_featured plus the search's own
  filters, ORDER BY RAND()); at 1.3M listings on dev36 it took 2.26 s of a
  2.5 s page, routed or not
- Verify: PostQueryRouterTest (key-only clause routes as IS NOT NULL),
  integration 590, unit 704, phpcs, stan; dev36 front-end
- PR: PR-122
2026-09-27 22:46:52 +08:00
wpdev eac0955b76 perf(2meet-data-optimizer): let flat-first sorts use a (column, id) index
- a group index may end with the entity ID column (post_id, ...)
- flat-first rewrites the posts.ID tie-breaks once, in the sort's direction:
  with a (column, post_id) index MySQL returns rows in order instead of
  sorting the whole match. dev36 (1.3M listings), price range sorted by price:
  850 ms filesort -> 2 ms. Ties are ordered by ID the same way as the column
  (natively ID DESC); pagination stays stable, result sets are unchanged
- Verify: EntityRegistryIndexesTest, PostQueryRouterFoundRowsTest (ASC
  tie-break, single tie-break), unit 704, integration 589, phpcs, stan;
  dev36 with the HivePress indexes (PR-119)
- PR: PR-118
2026-09-27 22:40:59 +08:00
wpdev b2725b219d test(2meet-data-optimizer): live tools no longer flush the object cache
- mut.php called wp_cache_flush() after each write and qbench.php
  wp_cache_flush_group() before each run; with Redis Object Cache's selective
  flush both are one server-side Lua scan over the site's keys, which blocked
  the shared Redis for 5–20 s on dev36 (3 M keys) and broke other sites'
  requests (BUSY / read errors)
- mut.php clears the touched post or user caches plus the runtime cache;
  qbench.php bumps the routed count version (posts' last_changed before 1.3.0)
- tests/ is not packaged; plugin code does not flush
2026-09-27 22:38:16 +08:00
wpdev ca2cbdab65 perf(2meet-data-optimizer): keep cached routed counts across unrelated post meta writes
- cached counts followed posts' last_changed, which WordPress moves on any
  post meta write; themes write meta on page views (on dev36 Blocksy saved a
  dynamic-CSS descriptor for each listing a crawler visited), so the cache
  was invalidated every few seconds and keyword/filter searches recounted
- counts now follow their own last_changed group (tmdo_posts), bumped on
  clean_post_cache (post saved or deleted; hooked in every mode) and on flat
  writes through the Hook Bus, plus terms' last_changed; a routed search reads
  no unmanaged meta
- Verify: PostQueryRouterFoundRowsTest (unrelated meta write keeps the count,
  save or flat write recounts), integration 588, unit 703, phpcs, stan;
  dev36 front-end
- PR: PR-117 (2.3 acceptance)
2026-09-27 17:46:54 +08:00
wpdev 7851f150fa perf(2meet-data-optimizer): key the routed count cache on the SQL without wpdb's placeholder escape
- keyword searches carry wpdb's placeholder escape for "%" ({<hash>}), which
  is new in every request, so md5(count SQL) never matched across requests
  and every keyword search recounted (the 2.2 "keyword + filter routed slower
  than native" finding). The key now uses remove_placeholder_escape()
- found in the 2.3 acceptance: dev36 keyword + price routed 6.2 s vs native
  4.9 s; PR-091's in-process check reused one process, one escape
- Verify: PostQueryRouterFoundRowsTest (two escapes, one COUNT), integration
  588, unit 703, phpcs, stan; dev36 front-end
- PR: PR-116 (2.3 acceptance)
2026-09-27 17:39:36 +08:00
wpdev 1d70d13f45 fix(2meet-data-optimizer): close the gaps in deferred table upgrades found in the 2.3 review
- a deferred group whose stored definition matches again (reverted, or
  upgraded by another process) goes through end_deferral() instead of staying
  native for good (review F2)
- the tmdo_ensure_group_indexes handler upgrades deferred tables itself: with
  ALTERNATE_WP_CRON the event runs after init in a visitor's request (F3)
- a new group table created while the entity writes to flat stays native
  until the drift fix has filled it from native meta; an opted-in HivePress
  attribute no longer routes searches to an empty table (F6)
- direct_read() returns native values for a deferred group (F10); the Hook
  Bus ignores deferral in aeav_only, as the router does
- dbDelta failures: the definition is recorded only when every column exists
  (schema.upgrade_failed otherwise), so a failed ALTER is retried and the
  group is not released onto missing columns
- test bootstrap: get_metadata() reads the real meta table
- Verify: SchemaUpgradeDeferralTest (6), Hook Bus direct read, integration
  587, unit 703, phpcs, stan
- PR: PR-114 (2.3 acceptance correctness review)
2026-09-27 17:30:36 +08:00
wpdev e26aadc4ec fix(2meet-data-optimizer): keep the audit trail of mode changes and snapshot deletions
- TMDO_Logger::notice() is always written to the error log (it had no
  callers); bridge_mode_changed and snapshot_deleted use it. Since PR-095
  info() is silent without WP_DEBUG_LOG, which left no record of who switched
  a mode or deleted a snapshot (security review L1)
- Verify: LoggerTest (notice always logged, info gated), unit 703,
  integration 585, phpcs, stan
- PR: PR-113 (2.3 acceptance security review)
2026-09-27 17:25:56 +08:00
wpdev 29adb05567 fix(2meet-data-optimizer): alter an existing flat table in one process at a time; keep a deferred group native until its drift fix
- security review H1: anonymous /wp-cron.php requests pass ddl_allowed(),
  and dbDelta on an existing table had no lock, so concurrent requests each
  ran the ALTER on a large table. The upgrade now runs under the same MySQL
  GET_LOCK as the index DDL (skipped when another process holds it) and
  re-checks the stored hash once locked; this covers aeav_only too. The table
  build moves unchanged to write_table(); lock()/unlock() are shared
- security review M1: after a deferred upgrade the group stays native until
  tmdo_drift_resync has rebuilt the rows written meanwhile
  (TMDO_Schema_Manager::resynced() from run_scheduled_fix()); with no flat
  writes (disabled) it is released at once
- Verify: SchemaUpgradeDeferralTest (lock held elsewhere, resync release),
  integration 585, unit 702, phpcs, stan
- PR: PR-111 (2.3 acceptance security review)
2026-09-27 17:22:57 +08:00
wpdev eb1c8b78f4 perf(2meet-data-optimizer): flush the cold cache group without a server-side scan
- TMDO_Cache_Layer::flush_group() bumps the post type's cold cache version
  (option wpdo_cold_cache_version) instead of wp_cache_flush_group(), which
  Redis Object Cache runs as one server-side scan that blocks Redis on a large
  group; entries under older keys expire by their TTL
- TMDO_Zone_Cold::cache_key() builds every cold blob key; version 0 keeps the
  current keys, so upgrading invalidates nothing
- Verify: CacheLayerTest (new keys, no scan, other post types untouched),
  unit 702, integration 583, phpcs, stan
- PR: PR-110
2026-09-27 17:12:14 +08:00
wpdev 06a3594417 fix(2meet-data-optimizer): take the index lock on the primary under read/write splitting
- when $wpdb has send_reads_to_masters() (HyperDB, LudicrousDB), it is called
  before SELECT GET_LOCK, which a replica would take without locking out the
  ALTER on the primary
- Verify: GroupCompositeIndexTest (call order through a wpdb proxy),
  integration 583, unit 701, phpcs, stan
- PR: PR-108
2026-09-27 17:09:10 +08:00
wpdev e18030b206 fix(2meet-data-optimizer): upgrade existing flat tables from WP-CLI or WP-Cron only
- a changed group definition no longer runs dbDelta on an existing table
  inside a web request (seconds to minutes on a large table, and every
  concurrent request tried it): the group is recorded in tmdo_schema_deferred
  and tmdo_ensure_group_indexes is scheduled, whose request upgrades it
- meanwhile the group stays native: Hook Bus field lookups return no field and
  the router treats it as drifting; afterwards a drift fix
  (tmdo_drift_resync) rebuilds the writes that skipped the flat table
- new tables are still created at once; aeav_only upgrades at once (the flat
  table is the only copy); doctor lists deferred groups
- Verify: SchemaUpgradeDeferralTest, Hook Bus and router tests, integration
  582, unit 701, phpcs, stan; dev36 live cycle in the 2.3 acceptance
- PR: PR-107
2026-09-27 17:07:48 +08:00
wpdev 5968bcd95d fix(2meet-data-optimizer): forget post status columns of groups whose flat table is gone
- on_transition() skips a recorded group whose table no longer exists and
  removes it from tmdo_post_status_columns; every post status change used to
  UPDATE the missing table and print a database error (seen on dev36 after a
  throwaway group's table was dropped)
- Verify: PostStatusSyncTest (dropped table), integration 578, unit 701, phpcs, stan
- PR: PR-106
2026-09-27 16:59:08 +08:00
wpdev a268470b60 fix(2meet-data-optimizer): treat contact and address keys as private in the public REST filter
- rest_exposable() also withholds keys named like an email, phone, address or
  IP, even when a field declares show_in_rest (defence in depth)
- Verify: RestApiTest (private and look-alike public keys), unit 701,
  integration 577, phpcs, stan; dev36 live rest suite
- PR: PR-096
2026-09-27 16:36:51 +08:00
wpdev 1bdd51d9a3 fix(2meet-data-optimizer): log info and notice events only when debug logging is on
- TMDO_Logger::info()/notice() write to the PHP error log only with
  WP_DEBUG_LOG (filter tmdo_log_info); every CLI run and production request
  printed [WPDO][INFO] lines
- debug() keeps its WP_DEBUG gate; warning() and error() are unchanged
- Verify: LoggerTest (2 new), unit 700, integration 577, phpcs, stan
- PR: PR-095
2026-09-27 16:35:13 +08:00
wpdev 22b46ad175 perf(2meet-data-optimizer): stop writing an option for every rate-limited view
- blocked POST /listings/{id}/view requests update wpdo_rl_stats 1 in 10,
  each sample adding 10: anonymous traffic no longer turns 1:1 into DB writes,
  and the admin top list stays representative (unbiased estimate)
- Verify: RestApiTest counts option writes (about 20 for 200 blocked requests),
  unit 698, integration 577, phpcs, stan
- PR: PR-094
2026-09-27 16:33:16 +08:00
wpdev b187c1c4d3 perf(2meet-data-optimizer): count routed searches from a cached base count minus the excluded posts
- a routed search with post__not_in (HivePress sends a random set per page)
  caches the COUNT without the exclusion and subtracts a live primary-key
  count of the excluded posts that match; the total stays exact
- the excluded count drops STRAIGHT_JOIN so MySQL starts from posts.ID
- Verify: PostQueryRouterFoundRowsTest, unit 698, integration 577, phpcs, stan;
  dev36 live router suite and keyword + price search
- PR: PR-091
2026-09-27 16:19:11 +08:00
wpdev ec8d69a6f4 fix(2meet-data-optimizer): add declared flat-table indexes from WP-CLI or WP-Cron only, one process at a time
- ensure_group_indexes() runs the ALTER only under WP-CLI or WP-Cron (filter
  tmdo_schema_ddl_allowed); web requests, including /wp-admin/ before
  auth_redirect() and admin-post.php, schedule tmdo_ensure_group_indexes instead
- MySQL GET_LOCK keeps it to one process (add_option() is INSERT ... ON
  DUPLICATE KEY UPDATE and cannot serve as a lock); released by MySQL if the
  process dies
- the applied signature is recorded only when every ALTER succeeds; failures
  log schema.index_failed and stay in pending_indexes(), which doctor reports
- new tables get their declared cidx_ indexes in CREATE TABLE
- test stub wpdb::query()/insert() return false on SQL errors like wpdb
- Verify: GroupCompositeIndexTest 7/7, unit 698, integration 576, phpcs, stan
- PR: PR-090a
2026-09-27 16:07:23 +08:00
wpdev 582ff13a11 test(2meet-data-optimizer): compare site-wide rating sorts by sort key, not id
Tests / PHP Lint (pull_request) Successful in 16s
Anti-EAV Lint + Quality Gate / anti-eav-lint (pull_request) Successful in 16s
Tests / Unit Tests (pull_request) Successful in 45s
Tests / PHPCS (pull_request) Successful in 35s
Tests / Integration Tests (pull_request) Successful in 1m38s
Tests / PHPStan (pull_request) Successful in 1m18s
- The PR-079 live router cases ran the HivePress rating sort on the whole site with an unqualified DECIMAL: native rounds 4.5 to 5 and orders its many ties by chance, so the id order passed or failed with the query plan. They now send DECIMAL(10,1), as the HivePress Rating field does, and compare each position's native sort key and found_posts; the flat-first / GROUP BY assertions stay.
- Verify: live router 18/18 twice; all suites 56/56
- PR: PR-089b
2026-09-27 10:20:30 +08:00
wpdev edf9e6bc93 fix(2meet-data-optimizer): doctor ignores drift records of groups no longer registered
- A group's last drift record outlived its registration (wc_product and user hot on dev36, any add-on deactivated): doctor counted it and could warn forever with a --fix hint that cannot reach it. Only records of backfillable groups are read; the router and clean_since already go by the registry.
- Verify: DoctorTest (RED before); dev36 doctor user 9 → 8 groups
- PR: PR-089a
2026-09-27 10:07:02 +08:00
wpdev 07955192d4 chore(2meet-data-optimizer): clean HivePress transients left in wp_options
- `wp tmdo cleanup-hp-transients --options [--dry-run|--confirm] [--batch=<n>]` removes the rows the HivePress AddOn wrote to wp_options (`_transient_wpdo_hp_pm_*` and their timeouts) before AddOn 0.3.0 kept them in the object cache, deleting in batches (default 5000). It works without the AddOn active; the namespace is repeated in core for that.
- Verify: CleanupHpTransientsTest (RED before; only the AddOn's rows go, look-alike names stay); dev36 --dry-run counts 2,314,750 rows in 3.2 s. The --confirm run belongs after the AddOn change is deployed.
- PR: PR-080
2026-09-27 09:06:19 +08:00
wpdev 84f6e32913 perf(2meet-data-optimizer): route sort-only searches from the flat table when every post has a row
- A sort-only search (HivePress _sort, no routed filter) LEFT JOINs the flat table to keep posts without a row; it now drives from the flat table (STRAIGHT_JOIN, INNER JOIN) when every post of the type has a flat row. The router gives a new post of a type with its own groups an empty row (wp_insert_post); coverage is one anti-join per group, cached 15 min and per post-mode change (forget_coverage()).
- A flat-first search drops the GROUP BY posts.ID WordPress adds for any tax query when the flat row is the only join (WooCommerce's product_visibility NOT IN on searches is a subquery); a joining tax query keeps it.
- Verify: live router suite (5 new cases, RED before); dev36 rating__desc page, ids identical: native ~3.3 s, 2.1 routing ~1.5–1.9 s, now ~0.51–0.85 s (main query 440 → 46 ms, COUNT 400 → 160 ms). p50 ≤ 300 ms is still not met: page rendering and HivePress's own get_posts() in pre_get_posts (suppress_filters, ~100 ms) remain.
- PR: PR-079
2026-09-27 08:31:47 +08:00
wpdev f48df1a33f refactor(2meet-data-optimizer): stop registering the wc_product post group
- WooCommerce reads product meta directly and writes stock by SQL, so a flat copy drifted (R-2.1 V1 T4); product meta now stays in native only (plan decision 2). TMDO_Post_Fields registers wp_core, attachment and nav_menu_item. Existing wp_wpdo_post_wc_product tables are left in place and still dropped on uninstall; the orphan restore registers nothing for them while native meta holds the keys; `wp tmdo bridge-restore-native post --group=wc_product --retire` archives them.
- Tests that route products register the former group themselves (tmdo_test_register_wc_product_group(), same name and table); live router Q06 runs on wp_core's _wp_page_template instead of _sku. The post stress tester already skips flat tables that do not exist.
- Verify: PostFieldsRegistrationTest (RED before); dev36: wc_product absent from the registry and not restored as an orphan, a CLI order took stock 452 → 450 in native with no DB error (reverted), drift post 10 groups 0.
- PR: PR-077
2026-09-27 01:11:52 +08:00
wpdev f15473527c fix(2meet-data-optimizer): stop false alarms in doctor, term-comment-diagnose and bridge-status
- doctor: entity flat tables come from the registry (TMDO_CLI::check_entity_tables()) instead of a hard-coded user list naming the never-registered sso group, which failed doctor on every site; it also reports Promotion Guard violations and the recorded drift status per entity (report_guard_and_drift()).
- term-comment-diagnose: filter status comes from each filter's is_enabled() (the garbage filter and misc bucket default to off but were shown enabled); a filter whose add-on is not active shows "not installed".
- bridge-status: flat_rows counts rows holding a value (wp_core keeps an all-NULL row per post: 15306% on dev36) and eav_ids of a post-type group counts only its post types.
- Tests: the WP_CLI stubs record their output in $GLOBALS['_wp_cli_output'].
- Verify: DoctorTest, CliFilterStatusTest, CliBridgeStatusTest (RED before); dev36 doctor exit 1 → 0, wp_core 15306% → 100%.
- PR: PR-076
2026-09-27 01:00:13 +08:00
wpdev 9d1a7c5591 feat(2meet-data-optimizer): throttle shadow_read diffs per key and promote on clean drift checks
- The shadow diff logger records at most 20 rows per entity type + key per hour across all objects, on top of the existing per-object 5-minute window: a key stale on every object (WooCommerce total_sales, written by direct SQL) no longer logs a row per product read.
- The auto-promoter's third condition is now TMDO_Drift_Check::clean_since(): every backfillable group of the entity checked within 2 days (sampled or full) with no drift; no record, no promotion. The shadow diff count no longer gates it (the filter's 4th argument is passed as 0). The Promotion Guard still refuses HivePress / WooCommerce sites.
- Reason classification was dropped (decision 2026-09-27): it needs a schema change and drift-check already reports miss / diff / extra per key.
- Verify: ShadowDiffThrottleTest, AutoPromoterDriftTest (RED before); dev36 simulated user shadow_read: clean_since yes, promotion still refused by the guard, mode unchanged.
- PR: PR-075
2026-09-27 00:48:18 +08:00
wpdev a26500206a feat(2meet-data-optimizer): backfill writes made while the Hook Bus was off
- When an entity goes from disabled to dual_write / shadow_read, or the Hook Bus flag is switched on, a single WP-Cron event (tmdo_drift_resync, one per entity, 60 s later) runs drift-check --fix for the entities that keep native meta, so values written while flat was bypassed land in flat. aeav_only is never touched.
- A full fix instead of the planned post_modified_gmt increment (decision 2026-09-27): meta-only writes such as WooCommerce's direct stock SQL do not bump post_modified. No wpdo_bridge_off_since.
- The resync hook is cleared on deactivate / uninstall (literal name: uninstall.php does not load TMDO_Drift_Check).
- Verify: DriftCheckTest resync cases (RED before); dev36 end to end with real WP-Cron: write while user disabled → miss 1 → bridge-mode user dual_write schedules tmdo_drift_resync [user] → run → drift 0.
- PR: PR-074
2026-09-27 00:41:43 +08:00
wpdev 1b21ee9add fix(2meet-data-optimizer): drift checks only where native meta is authoritative
- In aeav_only flat is the source of truth and native meta is stale by design, so drift there means nothing: drift-check --fix refuses (it would overwrite current flat values with stale native ones), the daily sample skips such entities, and the router keeps routing a drifting group instead of falling back to stale native meta.
- Found before PR-074 on hardening/2.2 (PR-071–073 not released; dev21 still runs aeav_only).
- Verify: DriftCheckTest / PostQueryRouterDriftTest aeav_only cases (RED before); dev36 fix() in a simulated aeav_only refused.
- PR: PR-073b
2026-09-27 00:34:03 +08:00
wpdev 6c782282c7 feat(2meet-data-optimizer): sample drift daily and record the result
- TMDO_Drift_Check::cron_tick() runs on the existing wpdo_daily_health_check event (no new schedule to add or clear): every entity type whose flat tables are written (dual_write or higher) is checked on 500 random entities per group, and the outcome replaces the recorded status, so a group found drifting leaves the router that day.
- Verify: DriftCheckTest cron cases (RED: method / hook missing); dev36 with all four entities in dual_write: one tick over 22 groups in 10 s, 0 drift, status option restored.
- PR: PR-073
2026-09-27 00:28:00 +08:00
wpdev ddefdc16d4 fix(2meet-data-optimizer): keep searches native while a group is known to drift
- The post query router routes no clause of a group whose last drift-check recorded miss / diff / extra; all or nothing, so such a query stays fully native. A group never checked keeps routing (decision 2026-09-27: no slowdown after upgrading; PR-073 samples daily).
- The check sits in route_clause(), which every routed clause, HivePress sort shape and meta_key sort goes through.
- Verify: PostQueryRouterDriftTest (drifting group and drifting sort key RED before); dev36 hp_listing price search in shadow_read: clean → flat, drift → native, unchecked → flat, found 90347 each.
- PR: PR-072
2026-09-27 00:22:47 +08:00
wpdev 3a347269c8 feat(2meet-data-optimizer): repair drifted flat rows from native with drift-check --fix
- TMDO_Drift_Check::fix() deletes flat rows of deleted entities and, for post groups, of posts outside the group's post types; rebuilds every other drifted entity's row from native through the new TMDO_Entity_Migration_Engine::migrate_entity() (keys without native rows become NULL, PR-066); keeps a row whose entity has no native value for the group but clears its columns (no NULL-only inserts, no deleted rows the router may join on); invalidates the row cache of each touched entity; then checks again. Native meta is never written.
- `wp tmdo drift-check <entity> --fix` (not combinable with --sample).
- Verify: DriftCheckTest fix cases (RED: method missing); dev36 --fix: post 42 rows rebuilt, 1 stray row (post 312, hp_booking in hp_draft_flags) deleted, all four entities 0 drift after, checker miss/extra 0.
- PR: PR-071
2026-09-27 00:07:33 +08:00
wpdev 02045c6f07 feat(2meet-data-optimizer): report flat-vs-native drift with wp tmdo drift-check
- New TMDO_Drift_Check compares every backfillable group of post / user / term / comment with native meta by value (first native row → safe unserialize → Type_Caster::to_db): miss / diff / extra per key, set-based SQL with BINARY exact matches, PHP comparison only for the remaining candidates. Serialized-vs-JSON and 50 vs 50.000000 are no longer drift; over-long text stored as NULL is not a miss; extra covers rows of other post types. Outcome per group goes to the wpdo_drift_status option; nothing else is written.
- `wp tmdo drift-check <entity> [--group] [--sample=<n>] [--format]`. A new class instead of extending the post-only shadow verifier, which queries row by row and writes shadow_diffs.
- Verify: DriftCheckTest; live suite drift; dev36 post miss=533 diff=2 extra=1 in 23 s (checker's 45 json false diffs gone), user/term/comment 0.
- PR: PR-070
2026-09-26 23:56:23 +08:00
wpdev 382786cb9d feat(2meet-data-optimizer): set an entity bridge mode from the CLI under the Promotion Guard
- New `wp tmdo bridge-mode <entity> <mode>` goes through TMDO_Mode_Manager::set(): unsafe jumps and aeav_only promotions refused by the Promotion Guard exit with the reason and leave the mode unchanged. Named apart from the Feature Flags `mode-set`.
- member-sso-sync and member-force-logout warn that they are deprecated: the user sso group they work on is no longer registered.
- Verify: CliBridgeModeTest (RED: method missing); dev36 bridge-mode user aeav_only refused (unsafe jump), term dual_write → shadow_read → aeav_only refused by the HivePress guard → back to disabled, wpdo_bridge_modes identical to before.
- PR: PR-069
2026-09-26 23:36:55 +08:00
wpdev 2022cf1d08 feat(2meet-data-optimizer): backfill every registered user group with member-backfill --all
- member-backfill / member-audit take their groups from the registry (TMDO_Entity_Migration_Engine::backfillable_groups(), restored orphans excluded) instead of a hard-coded list that still named the never-registered `sso`; `--all` backfills every user group (R-2.1 had to use wp eval). TMDO_Post_Migration::backfillable_groups() now delegates to it.
- member-cutover checks the Mode_Manager result with is_wp_error(): a guarded or invalid transition used to crash with a fatal (WP_Error used as an array).
- Verify: CliMemberBackfillTest (--all and the orphan filter RED before; the --group case cannot fail under the non-throwing WP_CLI stub); dev36 `member-backfill --all --reset` 9 groups, 0 errors, checker user miss/diff/extra 0; `member-cutover --confirm` exits 1 with the Guard message, mode unchanged.
- PR: PR-068
2026-09-26 23:30:28 +08:00
wpdev 7807eae957 refactor(2meet-data-optimizer): list post groups for backfill from the registry
- Admin "backfill all" and `wp tmdo post-migrate-group` take any registered post group via TMDO_Post_Migration::backfillable_groups() instead of a hard-coded list of 7 (dev36 registers 11; the 4 HP add-on groups were never backfilled). Restored orphan groups are excluded and rejected — their values live only in the flat table.
- Groups with json fields (attachment, nav_menu_item) go row by row through the migration engine instead of leaving the json columns NULL in the bulk pivot.
- Verify: PostBackfillJsonTest (3 RED on hardening/2.2); dev36 post-migrate-group attachment / nav_menu_item / hp_draft_flags: json columns semantically equal native (24/24, 21/21), checker unchanged.
- PR: PR-067
2026-09-26 22:54:27 +08:00
wpdev 13ea341ef8 fix(2meet-data-optimizer): do not invent default rows when migrating an entity group
- migrate_group() REPLACEs the whole flat row; columns without a native row fell back to their DDL default (session_count / account_flags DEFAULT 0), so every backfilled user got invented zeros. Keys with no native row are now written as NULL; required NOT NULL columns keep their default.
- Covers the cron batch path (migrate_group_batch) and clears stale flat values on re-run.
- Verify: EntityMigrationEngineDefaultsTest (3 RED on main); dev36 user/activity backfill: checker extra 123 → 0 for session_count and account_flags, miss/diff unchanged.
- PR: PR-066
2026-09-26 22:46:01 +08:00
wpdev 336664473b fix(2meet-data-optimizer): keep registered fields out of the public listings REST endpoints
Anti-EAV Lint + Quality Gate / anti-eav-lint (pull_request) Successful in 15s
Tests / PHP Lint (pull_request) Successful in 17s
Tests / Unit Tests (pull_request) Successful in 51s
Tests / Integration Tests (pull_request) Successful in 1m22s
Tests / PHPCS (pull_request) Successful in 1m12s
Tests / PHPStan (pull_request) Successful in 1m10s
- show_in_rest defaulted to true, so GET /wpdo/v1/listings and /listings/{id} (public by design) returned every registered hot / cold field to anonymous visitors — e.g. infocards tmeetic_ig_access_token_enc / _token_expiry / _user_id and WooCommerce _purchase_note (confirmed on dev36)
- Fields are private unless registered with show_in_rest => true; protected (underscore) and credential-like keys (token, secret, password, api key, private key, credential, salt, nonce) are never exposed
- Zone A list path no longer returns whole rows when nothing is public, and filters / orderby accept public columns only
- Version 1.1.1; CHANGELOG, readme
- Verify: unit 690 (+3), integration 525 (+3 private-column tests, RED on 1.1.0), live suite rest 2/2 (RED on 1.1.0), all live 49/49, phpcs 0, stan OK
2026-09-26 22:22:40 +08:00
wpdev dc9a8e9886 perf(2meet-data-optimizer): drive flat-sorted searches from the flat table
- A filtered search ordered by a flat column is rewritten to SELECT STRAIGHT_JOIN … FROM flat INNER JOIN wp_posts, with posts.ID tie-breaks (ours and plugin-appended) read as the equal flat post_id, so MariaDB sorts one table instead of looking every post up and sorting in a temporary table
- Only when the routed conditions already require a flat row on the sort group, without the copy route or a selective post filter; filter tmdo_router_flat_first turns it off; split_the_query compares with the same rewrite
- Verify: tests/live/run.php router (13/13); integration 522; dev36 page SQL Q07 263→147 ms, Q15 250→142 ms; fe.sh hourly_rate__desc 850→367 ms, ids == native
- PR: PR-062c
2026-09-26 21:05:11 +08:00
wpdev a7134bfb80 fix(2meet-data-optimizer): do not recreate a deleted entity's flat row
- Deleting a post / user / term / comment drops its flat row in the adapter delete hook, then WordPress deletes each meta row; sync_from_native upserted NULL for each and brought the row back (512 orphan rows on dev36). With no native value it now only clears the column on an existing row
- Verify: tests/live/run.php resync (25/25, entity deletes in dual_write and shadow_read leave no flat row); mut.php T8 flat_row NO-ROW; integration 516
- PR: PR-057b
2026-09-26 20:34:28 +08:00