🚧 Add import

README.md

@@ -115,6 +115,7 @@ Authentication is layered:
 | --- | --- | --- | --- | --- |
 | Read messages/devices/attributes/users | ✓ | ✓ | ✓ | ✓ |
 | Write devices, settings, attribute timelines, configs, tokens | — 403 | ✓ | ✓ | ✓ |
+| Import messages | — 403 | ✓ | ✓ | ✓ |
 | Add users | — 403 | ✓ — only role `user` | ✓ any role | ✓ |
 | Modify/delete users | — 403 | ✓ — only `user`-role accounts | ✓ (never the last admin) | ✓ |
 
@@ -123,8 +124,10 @@ The first admin is seeded from `BOOTSTRAP_ADMIN_USER`/`BOOTSTRAP_ADMIN_PASSWORD`
 | Endpoint | Auth | Description |
 | -------- | ---- | ----------- |
 | `GET /health` | none | Liveness probe (public) |
-| `GET /messages/count` | token | Total number of stored messages |
+| `GET /messages/count` | token | Total number of stored messages (unfiltered; the dashboard stat) |
 | `GET /messages` | token | List messages, newest first. The device's **current settings document** (from `imei_map`) is merged into each item's `payload`; on duplicate keys the stored payload wins. Each item also carries an **`attributes`** sibling field with the device's time-varying attributes **resolved as of the message's device time** (`payload_time`, `received_at` as the fallback) — apply the offsets to the payload values yourself (e.g. `payload[k] + attributes[k]`). Query parameters: `imei`, `device` (exact-match filters), `from`/`to` (ISO 8601 range on `received_at` — server receipt time; inclusive; naive values treated as UTC), `payload_from`/`payload_to` (same but on `payload_time` — the device timestamp extracted from the JSON; messages without one are excluded; ordering switches to `payload_time DESC`), `limit` (default 100, capped at `API_MAX_LIMIT`), `offset` for paging. The response carries a **filter-aware `total`** (the count over the same filters as `items`, so pagers can show "page X of Y" and jump to the last page) alongside `limit`/`offset`. Reversed ranges return 422 |
+| `GET /messages/export` | token | **Export selected messages as stored** (raw rows, no read-time settings merge, no resolved attributes). Query: repeatable `ids` (e.g. `?ids=1&ids=2`; at least one, capped at `API_MAX_LIMIT`). Returns a JSON array of `{id, received_at, topic, imei, device, payload, payload_time}` in the order the ids were given; unknown ids are skipped. Empty/oversized id lists → 422 |
+| `POST /messages/import` | superuser+ | **Import previously exported messages**: body is the JSON array from `GET /messages/export` (a single message object is accepted too). Rows are inserted as-is (original payload/timestamps/topic; `topic` defaults to `/import/<imei>`, absent `received_at` becomes now). The exported **`id` is ignored** — the database assigns fresh BIGSERIAL ids, so re-importing a file (or importing into the source database) never collides. Per-row validation → 422 with `row N` positions; max 10000 rows per request. Returns `{"status": "imported", "imported": n}` |
 | `GET /configs/{name}` | token | Read a config as JSON. `name`: `app_tokens` (secrets are not recoverable — app names with `null` placeholders) or `imei_map` (all device settings documents) |
 | `PUT/POST /configs/{name}` | token | **Bulk replace** a config in the database (delete + insert, transactional). Body **must be a JSON object**; per-config schema is validated (`imei_map` keys must be 15-digit IMEIs, values are settings documents — flat `"name"` values are promoted to `{"device": "name"}`; `app_tokens` must keep the `{"tokens": {name: secret}}` shape). Invalid payloads → 422, unknown names → 404. `app_tokens` changes take effect immediately; `imei_map` changes are picked up by the listener within ~5 seconds |
 | `GET /devices` | token | All devices, each as `{"imei": ..., **settings}` |
@@ -245,7 +248,7 @@ Example response:
 - [x] Remove clickable device links in tables for users that is not allowed to see the device view, or does not have enought authorization to be able to change anything. CLicking as a user ends up on a page with only some text. (→ resolved by the IMEI-first sweep: names are no longer links for anyone; the IMEI link leads to a legitimate read-only view for every role)
 - [ ] Change datetime localization. How is it handled now, from browser or static? I'm in sweden and not following swedish rules on layout.
 - [ ] Make it possible to manipulate a messages payload.
-- [ ] Make it possible, for superusers and admins, to manually inject new messages using json from an "insert message" view. It should allow array of json objects as well as a single message json object.
+- [x] Make it possible, for superusers and admins, to manually inject new messages using json from an "insert message" view. It should allow array of json objects as well as a single message json object. (→ the Messages view's **Import** dialog: paste JSON or load an exported file, single object or array both accepted; validated per row server-side. NOTE: payloads are stored AS GIVEN — no listener-style imei/device injection or timestamp extraction happens at import)
 - [ ] Change SESSION_TTL_DAYS to SESSION_TTL_MINUTES. Make SESSION_TTL_MINUTES being read from environmental variable. Default to 5 minutes.
 - [ ] Add user information. Name, mail address, localization
 - [ ] Add user settings view
@@ -254,10 +257,9 @@ Example response:
 - [ ] Change messages in tables to display information about the entire messege, not only the payload.
 - [X] Fix facet counts on Messages show 1 per device.
 - [x] Add ability to select how many rows to show in tables except the last messages table in the dashboard. Make it possible to select 10, 25, 50, 100, 200 or manual entry. Default to 25. (→ shared `Pager.vue` footer on Messages (server-side), Devices and Users (client-side via `useTable`); presets 10/25/50/100/200 + "custom…" manual entry clamped to `API_MAX_LIMIT`, default 25, persisted in localStorage; the dashboard's fixed 5-row list is untouched)
-- [ ] Add operations button in messages view for import/export or deletion of selected messages.
-- [ ] Export option will export selected messages as they are stored in the database, without added parameters/features.
-- [ ] Delete option will, after warning and user confirmation, delete the selected messages. Or will they be moved to retention table and held for 30 days?
-- [ ] Add ability to export/import original messages from message view. Possible to select all with one header selection checkbox.
+- [ ] Add operations button in messages view for import/export or deletion of selected messages. (→ export + import landed: per-row checkboxes + header select-all + "Export JSON" download; superuser+ "Import" dialog (paste/file, array or single object). Delete operation still open)
+- [x] Export option will export selected messages as they are stored in the database, without added parameters/features. (→ `GET /messages/export?ids=…` returns raw `mqtt_messages` rows — no settings merge, no attributes — and the UI downloads the JSON array; ids keep the caller's order)
+- [ ] Delete option will, after warning and user confirmation, move the deleted messages to retention. Where to see and be able to select messages to restore from retention? Rention lasts for 30 days.
 - [ ] Add payload timestamp as column in messages view. The from/to field would be able to select if it is received or payload timestamp that is used for framing the loaded messages.
 - [x] Add table navigastion buttons for jumping to first or last page. Add manual navigation textfield/dropdown to all tables exept dashboard last messages table. (→ the shared `Pager.vue`: « first / ‹ prev / page-number textfield (clamped, Enter or blur to apply) / › next / » last buttons + "page X of Y" + "n–m of total"; works on Messages (server-side offset/limit) and the client-side tables (Devices/Users via `useTable` pagination). `GET /messages` now returns a filter-aware `total` to power it)
 - [ ] Add versioning of UI and API, show versions on bottom left.