diff --git a/README.md b/README.md index 5b7a880..2790e0c 100644 --- a/README.md +++ b/README.md @@ -124,7 +124,7 @@ The first admin is seeded from `BOOTSTRAP_ADMIN_USER`/`BOOTSTRAP_ADMIN_PASSWORD` | -------- | ---- | ----------- | | `GET /health` | none | Liveness probe (public) | | `GET /messages/count` | token | Total number of stored messages | -| `GET /messages` | token | List messages, newest first. The device's **current settings document** (from `imei_map`) is merged into each item's `payload`; on duplicate keys the stored payload wins. Each item also carries an **`attributes`** sibling field with the device's time-varying attributes **resolved as of the message's device time** (`payload_time`, `received_at` as the fallback) — apply the offsets to the payload values yourself (e.g. `payload[k] + attributes[k]`). Query parameters: `imei`, `device` (exact-match filters), `from`/`to` (ISO 8601 range on `received_at` — server receipt time; inclusive; naive values treated as UTC), `payload_from`/`payload_to` (same but on `payload_time` — the device timestamp extracted from the JSON; messages without one are excluded; ordering switches to `payload_time DESC`), `limit` (default 100, capped at `API_MAX_LIMIT`), `offset` for paging. Reversed ranges return 422 | +| `GET /messages` | token | List messages, newest first. The device's **current settings document** (from `imei_map`) is merged into each item's `payload`; on duplicate keys the stored payload wins. Each item also carries an **`attributes`** sibling field with the device's time-varying attributes **resolved as of the message's device time** (`payload_time`, `received_at` as the fallback) — apply the offsets to the payload values yourself (e.g. `payload[k] + attributes[k]`). Query parameters: `imei`, `device` (exact-match filters), `from`/`to` (ISO 8601 range on `received_at` — server receipt time; inclusive; naive values treated as UTC), `payload_from`/`payload_to` (same but on `payload_time` — the device timestamp extracted from the JSON; messages without one are excluded; ordering switches to `payload_time DESC`), `limit` (default 100, capped at `API_MAX_LIMIT`), `offset` for paging. The response carries a **filter-aware `total`** (the count over the same filters as `items`, so pagers can show "page X of Y" and jump to the last page) alongside `limit`/`offset`. Reversed ranges return 422 | | `GET /configs/{name}` | token | Read a config as JSON. `name`: `app_tokens` (secrets are not recoverable — app names with `null` placeholders) or `imei_map` (all device settings documents) | | `PUT/POST /configs/{name}` | token | **Bulk replace** a config in the database (delete + insert, transactional). Body **must be a JSON object**; per-config schema is validated (`imei_map` keys must be 15-digit IMEIs, values are settings documents — flat `"name"` values are promoted to `{"device": "name"}`; `app_tokens` must keep the `{"tokens": {name: secret}}` shape). Invalid payloads → 422, unknown names → 404. `app_tokens` changes take effect immediately; `imei_map` changes are picked up by the listener within ~5 seconds | | `GET /devices` | token | All devices, each as `{"imei": ..., **settings}` | @@ -253,10 +253,11 @@ Example response: - [ ] Add view for viewing audit log. Place it with tokens and users under ACCESS menu section. - [ ] Change messages in tables to display information about the entire messege, not only the payload. - [X] Fix facet counts on Messages show 1 per device. -- [ ] Add ability to select how many rows to show in tables except the last messages table in the dashboard. Make it possible to select 10, 25, 50, 100, 200 or manual entry. Default to 25. +- [x] Add ability to select how many rows to show in tables except the last messages table in the dashboard. Make it possible to select 10, 25, 50, 100, 200 or manual entry. Default to 25. (→ shared `Pager.vue` footer on Messages (server-side), Devices and Users (client-side via `useTable`); presets 10/25/50/100/200 + "custom…" manual entry clamped to `API_MAX_LIMIT`, default 25, persisted in localStorage; the dashboard's fixed 5-row list is untouched) - [ ] Add operations button in messages view for import/export or deletion of selected messages. - [ ] Export option will export selected messages as they are stored in the database, without added parameters/features. - [ ] Delete option will, after warning and user confirmation, delete the selected messages. Or will they be moved to retention table and held for 30 days? - [ ] Add ability to export/import original messages from message view. Possible to select all with one header selection checkbox. - [ ] Add payload timestamp as column in messages view. The from/to field would be able to select if it is received or payload timestamp that is used for framing the loaded messages. -- [ ] Add table navigastion buttons for jumping to first or last page. Add manual navigation textfield/dropdown to all tables exept dashboard last messages table. \ No newline at end of file +- [x] Add table navigastion buttons for jumping to first or last page. Add manual navigation textfield/dropdown to all tables exept dashboard last messages table. (→ the shared `Pager.vue`: « first / ‹ prev / page-number textfield (clamped, Enter or blur to apply) / › next / » last buttons + "page X of Y" + "n–m of total"; works on Messages (server-side offset/limit) and the client-side tables (Devices/Users via `useTable` pagination). `GET /messages` now returns a filter-aware `total` to power it) +- [ ] Add versioning of UI and API, show versions on bottom left. \ No newline at end of file diff --git a/api/app.py b/api/app.py index 3980f55..2f4e8d5 100644 --- a/api/app.py +++ b/api/app.py @@ -1563,6 +1563,11 @@ def create_app( limit: int = Query(100, gt=0, description="Page size"), offset: int = Query(0, ge=0, description="Page offset"), ) -> dict[str, Any]: + """One page of messages. The response carries a filter-aware + "total" (the count over the SAME filters as the returned items) + next to limit/offset, so pagers can show "page X of Y" and support + jumping to the last page without a second request. GET + /messages/count stays the unfiltered total for the dashboard.""" if from_time is not None and to_time is not None and from_time > to_time: raise HTTPException( status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, @@ -1684,6 +1689,18 @@ def create_app( conn = psycopg.connect(config.dsn) try: with conn.cursor() as cur: + # Total matching the CURRENT filters, for the pager's + # page count / jump-to-last. Counting plain mqtt_messages + # (aliased m.) with the same WHERE is exact: the LEFT + # JOINs of the page query never remove rows. Same + # connection => same snapshot, so total and items can + # never disagree. + cur.execute( + sql.SQL("SELECT count(*) FROM mqtt_messages m") + where, + params, + ) + row = cur.fetchone() + total = row[0] if row is not None else 0 cur.execute(query, (*params, limit, offset)) rows = cur.fetchall() finally: @@ -1692,6 +1709,7 @@ def create_app( raise _db_unavailable(exc) from exc return { + "total": total, "limit": limit, "offset": offset, "from": from_time.isoformat() if from_time is not None else None, diff --git a/frontend/src/api.ts b/frontend/src/api.ts index f2decd1..ac37325 100644 --- a/frontend/src/api.ts +++ b/frontend/src/api.ts @@ -35,6 +35,9 @@ export interface MessageItem { } export interface MessagePage { + // total matching the CURRENT filters (not the unfiltered count) — feeds + // the pager's page count and last-page jump + total: number limit: number offset: number items: MessageItem[] diff --git a/frontend/src/components/Pager.vue b/frontend/src/components/Pager.vue new file mode 100644 index 0000000..ffa2c1c --- /dev/null +++ b/frontend/src/components/Pager.vue @@ -0,0 +1,199 @@ + + + + + \ No newline at end of file diff --git a/frontend/src/composables/useTable.ts b/frontend/src/composables/useTable.ts index 5e26f7f..2f1d653 100644 --- a/frontend/src/composables/useTable.ts +++ b/frontend/src/composables/useTable.ts @@ -7,7 +7,9 @@ // sort: 'string' | 'number' (default 'string') // filter: 'text' | 'facet' | 'text+facet' (default: no filtering) -import { computed, reactive, ref, type Ref } from 'vue' +import { computed, reactive, ref, watch, type Ref } from 'vue' + +import { loadPageSize, savePageSize } from '../prefs' export type FilterKind = 'text' | 'facet' | 'text+facet' @@ -87,6 +89,31 @@ export function useTable(rows: Ref, columns: Record + sortedFilteredRows.value.slice(offset.value, offset.value + limit.value), + ) + // The last page's offset is floor((n-1)/limit)*limit - NOT n-limit, which + // would land mid-page (60 rows / 25 -> 35 instead of 50). + watch(sortedFilteredRows, (rows) => { + const last = rows.length + ? Math.floor((rows.length - 1) / limit.value) * limit.value + : 0 + if (offset.value > last) offset.value = last + }) + watch(limit, (v) => savePageSize(v)) + + function setPageSize(n: number): void { + // keep the current top row visible across the size change + offset.value = Math.floor(offset.value / n) * n + limit.value = n + } + function clearColumn(name: string): void { const st = state[name] if (!st) return @@ -106,6 +133,10 @@ export function useTable(rows: Ref, columns: Record 0) and inside +// the server cap API_MAX_LIMIT (config.max_limit, default 10000). +export const MAX_LIMIT = 10000 + +export function loadPageSize(): number { + const raw = localStorage.getItem(PAGE_SIZE_KEY) + const n = raw === null ? NaN : Number(raw) + if (Number.isInteger(n) && n >= 1 && n <= MAX_LIMIT) return n + return 25 +} + +export function savePageSize(n: number): void { + localStorage.setItem(PAGE_SIZE_KEY, String(n)) +} \ No newline at end of file diff --git a/frontend/src/style.css b/frontend/src/style.css index 42983d2..0dd2c28 100644 --- a/frontend/src/style.css +++ b/frontend/src/style.css @@ -309,6 +309,11 @@ button:disabled { flex-wrap: wrap; } +.pager { + margin-top: 0.75rem; + gap: 0.75rem; +} + .grow { flex: 1; } diff --git a/frontend/src/views/Devices.vue b/frontend/src/views/Devices.vue index a3572ef..49dacaa 100644 --- a/frontend/src/views/Devices.vue +++ b/frontend/src/views/Devices.vue @@ -5,6 +5,7 @@ import { api } from '../api' import type { Device } from '../api' import { useTable } from '../composables/useTable' import TableHeader from '../components/TableHeader.vue' +import Pager from '../components/Pager.vue' import { canEdit, showError, toast } from '../store' const devices = ref([]) @@ -121,7 +122,7 @@ async function removeDevice(d: Device) { - + {{ d.imei }} @@ -143,5 +144,13 @@ async function removeDevice(d: Device) {
+
\ No newline at end of file diff --git a/frontend/src/views/Messages.vue b/frontend/src/views/Messages.vue index 8736bd5..d4cec11 100644 --- a/frontend/src/views/Messages.vue +++ b/frontend/src/views/Messages.vue @@ -5,12 +5,16 @@ import { api } from '../api' import type { Device, MessagePage } from '../api' import { formatTime } from '../format' import { showError } from '../store' +import { loadPageSize } from '../prefs' import TableHeader from '../components/TableHeader.vue' +import Pager from '../components/Pager.vue' const page = ref(null) const loading = ref(false) const offset = ref(0) -const limit = 25 +// persisted global page size (prefs.ts); the server caps it at +// API_MAX_LIMIT (default 10000) +const limit = ref(loadPageSize()) // the full device population for the facet lists (not just the loaded page) const devices = ref([]) @@ -35,7 +39,7 @@ const to = ref('') function buildQuery(): string { const params = new URLSearchParams() - params.set('limit', String(limit)) + params.set('limit', String(limit.value)) params.set('offset', String(offset.value)) if (sort.value) params.set('sort', sort.value) for (const v of imeiChecked.value) params.append('imei', v) @@ -47,17 +51,34 @@ function buildQuery(): string { return `/api/messages?${params.toString()}` } +// Server-side loads can overlap (e.g. the Pager's size change updates offset +// and limit at once). A monotonically increasing token makes stale responses +// harmless: only the newest request may write page.value. +let loadSeq = 0 + async function load() { + const seq = ++loadSeq loading.value = true try { - page.value = await api(buildQuery()) + const body = await api(buildQuery()) + if (seq !== loadSeq) return // a newer request superseded this one + page.value = body } catch (e) { - showError(e) + if (seq === loadSeq) showError(e) } finally { - loading.value = false + if (seq === loadSeq) loading.value = false } } +// Coalesces a burst of state updates (a Pager size change emits BOTH +// update:offset and update:limit) into one request built from the FINAL +// state — no intermediate request with a half-updated offset/limit. +let reloadTimer: number | undefined +function loadSoon(): void { + window.clearTimeout(reloadTimer) + reloadTimer = window.setTimeout(load, 0) +} + function applyFilters() { offset.value = 0 load() @@ -104,14 +125,18 @@ function onText(kind: 'imei' | 'device', text: string) { textTimer = window.setTimeout(applyFilters, 400) } -function prev() { - offset.value = Math.max(0, offset.value - limit) - load() +// Pager callbacks: a size change emits BOTH update:offset (position kept, +// top row stays visible) and update:limit — loadSoon() merges them into a +// single request; a lone offset is a plain page jump (also coalesced, so +// fast clicking cannot queue one request per click). +function onOffset(newOffset: number): void { + offset.value = newOffset + loadSoon() } -function next() { - offset.value += limit - load() +function onLimit(newLimit: number): void { + limit.value = newLimit + loadSoon() } onMounted(async () => { @@ -228,13 +253,14 @@ onMounted(async () => { -
- - items {{ offset + 1 }}–{{ offset + (page?.items.length ?? 0) }} - -
+
\ No newline at end of file diff --git a/frontend/src/views/Users.vue b/frontend/src/views/Users.vue index c140f50..891917d 100644 --- a/frontend/src/views/Users.vue +++ b/frontend/src/views/Users.vue @@ -4,6 +4,7 @@ import { Plus, Trash2, UserRound } from 'lucide-vue-next' import { api } from '../api' import { useTable } from '../composables/useTable' import TableHeader from '../components/TableHeader.vue' +import Pager from '../components/Pager.vue' import { canEdit, isAdmin, showError, toast, ui } from '../store' const users = ref<{ username: string; role: string }[]>([]) @@ -143,7 +144,7 @@ async function removeUser(u: { username: string; role: string }) { - + {{ u.username }} {{ u.role }} @@ -164,5 +165,13 @@ async function removeUser(u: { username: string; role: string }) { + \ No newline at end of file