Add data initi script
Tests / backend-test (pull_request) Successful in 6s
Tests / frontend-test (pull_request) Failing after 8s
Tests / e2e-test (pull_request) Failing after 1m30s

This commit is contained in:
(jenkins)
2026-04-17 00:09:00 +01:00
parent eda4697b03
commit 394394b387
3 changed files with 77 additions and 0 deletions
+11
View File
@@ -68,6 +68,17 @@
---
## 📊 Data Management
The project includes a compressed dataset of ~68,000 cities from Natural Earth.
| Action | Command |
| :--- | :--- |
| **Import/Refresh Data** | `./import-cities.sh` |
*Note: The script will truncate the `cities` table and perform a fresh import of the global dataset stored in `docker/data/cities.csv.gz`.*
---
## 📝 Roadmap Highlights
- [ ] Implement real ST_DWithin() PostGIS queries in the backend.
- [ ] Import full Natural Earth/GeoNames datasets into the `cities` table.
Binary file not shown.
+66
View File
@@ -0,0 +1,66 @@
#!/bin/bash
# Configuration
CONTAINER_NAME="line-of-sight-db"
DB_USER="line_of_sight"
DB_NAME="line_of_sight"
DATA_FILE="./docker/data/cities.csv.gz"
# Check if docker is available using the known path
DOCKER_BIN="/usr/local/bin/docker"
if [ ! -f "$DOCKER_BIN" ]; then
DOCKER_BIN=$(which docker)
fi
if [ -z "$DOCKER_BIN" ]; then
echo "Error: docker command not found."
exit 1
fi
# Check if data file exists
if [ ! -f "$DATA_FILE" ]; then
echo "Error: Data file $DATA_FILE not found."
exit 1
fi
# Check if container is running
if ! "$DOCKER_BIN" ps | grep -q "$CONTAINER_NAME"; then
echo "Error: Container $CONTAINER_NAME is not running."
echo "Please run 'docker-compose up -d' first."
exit 1
fi
echo "🚀 Starting database initialization..."
# Use a HEREDOC to execute multi-line SQL
# We load into a temp table first to handle the WKT -> Geography conversion
IMPORT_SQL=$(cat <<EOF
CREATE TEMP TABLE tmp_cities (
name VARCHAR(255),
population INTEGER,
country VARCHAR(100),
geom_wkt TEXT
);
COPY tmp_cities FROM STDIN WITH CSV HEADER;
TRUNCATE cities;
INSERT INTO cities (name, population, country, geom)
SELECT
name,
population,
country,
ST_GeomFromText(geom_wkt, 4326)::geography
FROM tmp_cities;
DROP TABLE tmp_cities;
SELECT count(*) || ' cities successfully imported.' as result FROM cities;
EOF
)
# Stream the gzipped data into psql
gunzip -c "$DATA_FILE" | "$DOCKER_BIN" exec -i -e PGPASSWORD=line_of_sight_pass "$CONTAINER_NAME" psql -U "$DB_USER" -d "$DB_NAME" -h 127.0.0.1 -c "$IMPORT_SQL"
echo "✅ Initialization complete."