Configurable Manticore Search backend for phpBB with multilingual tokenization.
  • PHP 95.5%
  • Shell 3.5%
  • JavaScript 0.8%
  • HTML 0.2%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-09-27 08:45:22 +00:00
adm/style feat: add configurable multilingual search tokenization 2026-09-27 08:44:47 +00:00
config add AI Candidate Search 2026-07-12 07:54:18 +00:00
core feat: add configurable multilingual search tokenization 2026-09-27 08:44:47 +00:00
docs feat: add configurable multilingual search tokenization 2026-09-27 08:44:47 +00:00
event add AI Candidate Search 2026-07-12 07:54:18 +00:00
examples/docker feat: add configurable multilingual search tokenization 2026-09-27 08:44:47 +00:00
language feat: add configurable multilingual search tokenization 2026-09-27 08:44:47 +00:00
migrations feat: add configurable multilingual search tokenization 2026-09-27 08:44:47 +00:00
search feat: add configurable multilingual search tokenization 2026-09-27 08:44:47 +00:00
styles/all/template/event style: render grouped topic search link 2026-05-27 07:22:41 +08:00
tests feat: add configurable multilingual search tokenization 2026-09-27 08:44:47 +00:00
.gitignore add release script 2026-05-27 21:17:56 +08:00
build-release.sh remove old build artifact when calling the build script 2026-05-28 23:22:36 +08:00
composer.json v0.5.0 2026-09-27 08:45:22 +00:00
ext.php chore: convert skeleton metadata 2026-05-27 01:18:15 +08:00
license.txt chore: establish extension baseline 2026-05-27 01:16:00 +08:00
phpunit.xml.dist test: add task 6 phpunit suite 2026-05-27 02:08:02 +08:00
README.md feat: add configurable multilingual search tokenization 2026-09-27 08:44:47 +00:00

Manticore Search for phpBB

Configurable Manticore Search backend for phpBB 3.3, with language-specific stemming, optional Chinese Jieba segmentation, and CJK character-based indexing.

Requirements

  • phpBB 3.3
  • PHP 8.1 or newer
  • PHP json extension
  • Manticore Search HTTP API reachable from phpBB, on the configured host and HTTP port (default: 9308)
  • Optional: OpenCC CLI for Simplified/Traditional Chinese query expansion

Manticore Docker Example

A standalone Manticore service and sample Jieba user dictionary are included in examples/docker. From the extension directory, run:

docker compose -f examples/docker/compose.yaml up -d

See the Docker example instructions for connection settings, persistent storage, and dictionary customization. You can also use an existing Manticore server. Jieba is optional and requires manticore-language-packs on that server.

Install In phpBB

Copy the extension into ext/bobmaster/manticoresearch under your phpBB root (the directory containing config.php and bin/phpbbcli.php). The extension's composer.json should be directly inside that directory.

Enable Manticore Search in ACP > Customise > Manage extensions, then purge phpBB's cache. Alternatively, run these commands from your phpBB root:

php bin/phpbbcli.php extension:enable bobmaster/manticoresearch
php bin/phpbbcli.php cache:purge

ACP Configuration

Open ACP > General > Search settings and select Manticore Search as the search backend. The default extension settings are:

  • Host: 127.0.0.1
  • Port: 9308
  • Table: phpbb_posts
  • Timeout: 5
  • Maximum matches: 20000
  • Charset table: non_cont
  • Morphology: empty (no stemming)
  • N-gram characters: empty; n-gram length: 0 (disabled)
  • Minimum word length: 1
  • Jieba user dictionary: empty (built-in dictionary when Jieba is selected)
  • OpenCC query expansion: disabled

When using the Docker example with PHP running on the host, keep 127.0.0.1:9308. If PHP runs in Docker, connect both services to the same Docker network and use manticore:9308; 127.0.0.1 inside the PHP container refers to that container itself.

New installations use these neutral defaults. Upgrades preserve existing language, dictionary, and OpenCC settings; the migration only adds the n-gram settings. See Language configuration for English, German, French, Chinese, Japanese, Korean, and mixed CJK examples.

Jieba options appear when morphology includes jieba_chinese. The OpenCC switch appears when the configuration includes Chinese-capable character sets or morphology, or when OpenCC is already enabled. Explicit Unicode mappings also expose the switch because they may include Chinese characters. OpenCC command and conversion settings appear only while the switch is enabled. These controls update as you edit with JavaScript; without JavaScript, save to refresh visibility. Hiding fields preserves their values and does not automatically enable features.

The backend creates the Manticore real-time table during indexing. It does not implement create_index(), so phpBB's ACP Search index page calls index() once per post during rebuilds.

Simplified And Traditional Chinese Expansion

When enabled in the ACP, the extension can expand Chinese keyword searches through OpenCC at query time. It keeps the index schema unchanged and searches the original query plus up to two converted variants:

  • Simplified to Traditional: default OpenCC config s2t.json
  • Traditional to Simplified: default OpenCC config t2s.json

If OpenCC is not installed or a conversion fails, the backend silently searches only the original query. This keeps search available even when OpenCC is missing.

Install OpenCC through the operating system package manager, then check that the PHP process can run it:

opencc --version
printf '结巴分词' | opencc -c s2t.json
printf '結巴分詞' | opencc -c t2s.json

The ACP settings can override the OpenCC command and config names.

Rebuild The Index

In ACP > Maintenance > Search index:

  1. Delete the Manticore Search index if it exists.
  2. Create the Manticore Search index.
  3. Let the phpBB progress page finish.

To verify the index through Manticore HTTP, adjust the host, port, and table name below to match your deployment:

curl -s http://127.0.0.1:9308/sql \
	--data-urlencode "query=SELECT COUNT(*) count FROM phpbb_posts"

Search Smoke Checks

Check a term present in your indexed posts (replace example below):

curl -s http://127.0.0.1:9308/sql \
	--data-urlencode "query=SELECT id, post_subject FROM phpbb_posts WHERE MATCH('example') LIMIT 5"

Open your forum's search page (search.php) and search for the same term. For language-specific checks, use the appropriate language configuration before rebuilding.

The extension registers bobmaster.manticoresearch.ai_candidate_search for server-side AI retrieval extensions. This is a phpBB dependency-injection service ID, using the vendor.extension.service naming convention: bobmaster is the vendor, manticoresearch is the extension, and ai_candidate_search is the service. The prefix prevents collisions with other extensions and core services; it is part of the extension identity, not a local environment setting. See the phpBB dependency injection documentation.

It returns post IDs and relevance scores, never indexed post bodies. Queries use proximity_bm25, title weighting, OpenCC expansion, quorum matching for longer natural-language queries, and a positive forum allowlist supplied by the caller. With n-grams enabled, each whitespace-delimited keyword is a phrase; character-level quorum matching is disabled to avoid matching unrelated combinations of characters.

bobmaster/manticoresearch also synchronizes indexed visibility and location attributes after phpBB soft-delete, restore, topic move, and post move events. After upgrading an existing installation, rebuild the Manticore index once from ACP > Maintenance > Search index to repair attributes that became stale before these listeners were installed.

Tests

With phpBB development dependencies installed, run from the phpBB root:

vendor/bin/phpunit -c ext/bobmaster/manticoresearch/phpunit.xml.dist

Run PHP syntax checks from the extension directory:

find . \
	-path '*/vendor' -prune -o -name '*.php' -print0 | xargs -0 -n1 php -l

An optional integration test creates temporary tables on a test Manticore instance, checks multiple languages through the backend and AI retrieval service, and removes the tables afterwards:

MANTICORE_TEST_HOST=127.0.0.1 MANTICORE_TEST_PORT=9308 php tests/integration/language_search.php

The test includes Jieba and libstemmer examples, so use a Manticore build with those features (the Docker example image was tested).