Links

Full-Text Search

Client-side full-text search (FTS) is available using the SQLite FTS5 extension.
This requires creating a separate FTS5 table(s) to index the data, and updating the table(s) using SQLite triggers.
Note that the availability of FTS is dependent on the underlying sqlite package used, as it is an extension that must first be enabled in the package.
Currently, FTS will work out the box in:
  1. 1.

    powersync.dart (PowerSync Flutter/Dart SDK)

We have an example in our Flutter To-Do List Demo App which is explained below.

Flutter To-Do List Demo App Example

Migrations

Creation of the FTS tables is initiated in powersync.dart while opening the database (DB).
Future<void> openDatabase() async {
...
await configureFts(db);
}
First, we need to set up the FTS tables to match the lists and todos tables already created in this demo app. We use the sqlite_async Dart package to generate these migrations. Don't worry if you already have data in the tables, as it will be copied into the new FTS tables.
The migrations are run in migrations/fts_setup.dart
/// This is where you can add more migrations to generate FTS tables
/// that correspond to the tables in your schema and populate them
/// with the data you would like to search on
Future<void> configureFts(PowerSyncDatabase db) async {
migrations
..add(createFtsMigration(
migrationVersion: 1,
tableName: 'lists',
columns: ['name'],
tokenizationMethod: 'porter unicode61'))
..add(createFtsMigration(
migrationVersion: 2,
tableName: 'todos',
columns: ['description', 'list_id'],
));
await migrations.migrate(db);
}
The createFtsMigration function corresponds to the below in migrations/fts_setup.dart
/// Create a Full Text Search table for the given table and columns
/// with an option to use a different tokenizer otherwise it defaults
/// to unicode61. It also creates the triggers that keep the FTS table
/// and the PowerSync table in sync.
SqliteMigration createFtsMigration(
{required int migrationVersion,
required String tableName,
required List<String> columns,
String tokenizationMethod = 'unicode61'}) {
String internalName =
schema.tables.firstWhere((table) => table.name == tableName).internalName;
String stringColumns = columns.join(', ');
return SqliteMigration(migrationVersion, (tx) async {
// Add FTS table
await tx.execute('''
CREATE VIRTUAL TABLE IF NOT EXISTS fts_$tableName
USING fts5(id UNINDEXED, $stringColumns, tokenize='$tokenizationMethod');
''');
// Copy over records already in table
await tx.execute('''
INSERT INTO fts_$tableName(rowid, id, $stringColumns)
SELECT rowid, id, ${generateJsonExtracts(ExtractType.columnOnly, 'data', columns)}
FROM $internalName;
''');
// Add INSERT, UPDATE and DELETE and triggers to keep fts table in sync with table
await tx.execute('''
CREATE TRIGGER IF NOT EXISTS fts_insert_trigger_$tableName AFTER INSERT
ON $internalName
BEGIN
INSERT INTO fts_$tableName(rowid, id, $stringColumns)
VALUES (
NEW.rowid,
NEW.id,
${generateJsonExtracts(ExtractType.columnOnly, 'NEW.data', columns)}
);
END;
''');
await tx.execute('''
CREATE TRIGGER IF NOT EXISTS fts_update_trigger_$tableName AFTER UPDATE
ON $internalName BEGIN
UPDATE fts_$tableName
SET ${generateJsonExtracts(ExtractType.columnInOperation, 'NEW.data', columns)}
WHERE rowid = NEW.rowid;
END;
''');
await tx.execute('''
CREATE TRIGGER IF NOT EXISTS fts_delete_trigger_$tableName AFTER DELETE
ON $internalName BEGIN
DELETE FROM fts_$tableName WHERE rowid = OLD.rowid;
END;
''');
});
}
After this is run, you should have the following tables and triggers in your SQLite DB:
FTS tables and migrations
FTS triggers

FTS Search Delegate

To show off this new functionality, we have created a search bar at the top of the screen in the To-Do List demo app:
Clicking on the search icon will open a search bar which will allow you to search for lists or todos that you have generated.
Example of searching
It uses a custom search delegate widget found in widgets/fts_search_delegate.dart to display the search results.

FTS Helper

We added a helper in fts_helpers.dart that allows you to add additional search functionality which can be found in SQLite FTS5 extension documentation.
String _createSearchTermWithOptions(String searchTerm) {
// adding * to the end of the search term will match any word that starts with the search term
// e.g. searching bl will match blue, black, etc.
// consult FTS5 Full-text Query Syntax documentation for more options
String searchTermWithOptions = '$searchTerm*';
return searchTermWithOptions;
}
/// Search the FTS table for the given searchTerm and return results ordered by the
/// rank of their relevance
Future<List> search(String searchTerm, String tableName) async {
String searchTermWithOptions = _createSearchTermWithOptions(searchTerm);
return await db.execute(
'SELECT * FROM fts_$tableName WHERE fts_$tableName MATCH ? ORDER BY rank',
[searchTermWithOptions]);
}