Ilker ManapandClaude Opus 5
b6de469351
CI / test (push) Has been cancelled
Keys are stored hashed, so a migration that drops them cannot recreate them: every person and every script needs a new one issued, and the old ones cannot even be looked up to say which they were. `llmgw keys move-store` copies instead. Found by running it against the real cluster, twice, after the gateway had already been stopped. The first order came from the order the migrations create tables in, and users is created in a later migration than api_keys even though api_keys references it — so the copy died three tables in. The order is now checked by a test that derives the foreign keys from the migrations rather than restating them, so a new reference fails there instead of during somebody's migration. Checked by putting the wrong order back and watching it fail. Then key_claims. Its sealed column is emptied the moment a claim is used, so every claimed row holds an empty blob in a NOT NULL column. An empty blob scans as a typed nil slice and a genuine NULL as an untyped one — a distinction this side can see and pgx cannot, since it encodes any nil slice as NULL. Restored before the insert, which is the difference between a copy that completes and one that stops four tables in. One transaction per table rather than one for the whole copy: a single transaction holding every row of a large usage table is how a migration runs out of memory on the machine least able to spare it. A failure therefore leaves a partial copy, which is why the destination must have been empty — starting again is dropping and recreating it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
72 lines
2.4 KiB
Go
72 lines
2.4 KiB
Go
package cli
|
|
|
|
import (
|
|
"fmt"
|
|
|
|
"github.com/spf13/cobra"
|
|
|
|
"gitea.mynodes.xyz/ilker/llmgw/internal/store"
|
|
)
|
|
|
|
// newMoveStoreCmd copies a store into another one.
|
|
//
|
|
// Written because keys are stored hashed. Moving to PostgreSQL without this
|
|
// means every person and every script needs a new key issued, and the old ones
|
|
// cannot even be looked up to say which they were.
|
|
//
|
|
// It reads and writes, and nothing else runs while it does: run it with the
|
|
// gateway stopped. A copy taken while requests are landing would miss whatever
|
|
// arrived after the table it had already passed.
|
|
func newMoveStoreCmd() *cobra.Command {
|
|
var from, to string
|
|
|
|
cmd := &cobra.Command{
|
|
Use: "move-store",
|
|
Short: "Copy a store into another one, for moving SQLite to PostgreSQL",
|
|
Long: "Copies every table, parents first, into an empty destination.\n\n" +
|
|
"Stop the gateway first. This makes no attempt to copy a store that is\n" +
|
|
"being written to: rows landing after the table they belong to has been\n" +
|
|
"passed would simply be absent, and nothing would say so.\n\n" +
|
|
"The destination must be empty. Merging two stores means deciding which\n" +
|
|
"of two projects with the same id was meant, and there is no answer to\n" +
|
|
"that which is not a guess.",
|
|
Args: cobra.NoArgs,
|
|
RunE: func(cmd *cobra.Command, _ []string) error {
|
|
if from == "" || to == "" {
|
|
return fmt.Errorf("--from and --to are both required")
|
|
}
|
|
src, err := store.Open(cmd.Context(), from)
|
|
if err != nil {
|
|
return fmt.Errorf("opening the source: %w", err)
|
|
}
|
|
defer src.Close()
|
|
|
|
dst, err := store.Open(cmd.Context(), to)
|
|
if err != nil {
|
|
return fmt.Errorf("opening the destination: %w", err)
|
|
}
|
|
defer dst.Close()
|
|
|
|
res, err := src.CopyTo(cmd.Context(), dst)
|
|
// Printed even on failure: a partial copy is the one case where
|
|
// knowing how far it got decides what to do next.
|
|
out := cmd.OutOrStdout()
|
|
var total int64
|
|
for _, r := range res {
|
|
if r.Rows > 0 {
|
|
fmt.Fprintf(out, " %-16s %d\n", r.Table, r.Rows)
|
|
}
|
|
total += r.Rows
|
|
}
|
|
if err != nil {
|
|
return err
|
|
}
|
|
fmt.Fprintf(out, "\n%d rows copied. Point the gateway at the new store and start it.\n", total)
|
|
return nil
|
|
},
|
|
}
|
|
cmd.Flags().StringVar(&from, "from", "", "source: a SQLite path, a postgres:// URL, or @file holding one")
|
|
cmd.Flags().StringVar(&to, "to", "", "destination, same forms")
|
|
return cmd
|
|
}
|