Data Sources
Sightline supports multiple data sources simultaneously, allowing you to aggregate incidents from different providers. This guide covers how to configure and work with each adapter type.
Overview
Data sources are configured in map.config.ts:
dataSources: [
{
id: 'source-1',
name: 'Source One',
adapter: 'static',
adapterConfig: { path: '/data/source1.json' },
enabled: true,
},
{
id: 'source-2',
name: 'Source Two',
adapter: 'supabase',
adapterConfig: {},
enabled: true,
},
]Users can toggle individual sources on/off using the source toggle control.
Static JSON Adapter
The simplest way to get started. Load incidents from a JSON file.
Setup
- Create a JSON file in
public/data/:
{
"incidents": [
{
"id": "incident-001",
"status": "published",
"createdAt": "2024-01-15T00:00:00Z",
"updatedAt": "2024-01-15T00:00:00Z",
"temporal": {
"date": "2024-01-15",
"dateCertainty": "exact"
},
"location": {
"name": "Phoenix, Arizona",
"country": "United States",
"latitude": 33.4484,
"longitude": -112.074,
"siteType": "urban",
"locationSensitivity": "standard"
},
"summary": "Description of the incident."
}
]
}- Configure the adapter:
{
id: 'my-data',
name: 'My Incidents',
adapter: 'static',
adapterConfig: {
path: '/data/my-incidents.json',
},
enabled: true,
}Best Practices
- Keep files under 5MB for optimal performance
- Use meaningful IDs for incidents
- Validate JSON before deploying
Supabase Adapter
Connect to a Supabase PostgreSQL database for production deployments.
Database Setup
-
Create a Supabase project at supabase.com
-
Create the incidents table:
CREATE TABLE incidents (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
status TEXT NOT NULL DEFAULT 'draft',
created_at TIMESTAMPTZ DEFAULT NOW(),
updated_at TIMESTAMPTZ DEFAULT NOW(),
-- Temporal
temporal JSONB NOT NULL,
-- Location
location JSONB NOT NULL,
-- Optional sections
summary TEXT,
classification JSONB,
witnesses JSONB,
sensor_evidence JSONB,
object_characteristics JSONB,
movement JSONB,
investigation JSONB,
response_impact JSONB,
source_data JSONB,
environment JSONB,
media JSONB,
related_incidents JSONB
);
-- Enable RLS
ALTER TABLE incidents ENABLE ROW LEVEL SECURITY;
-- Public read access for published incidents
CREATE POLICY "Public read access"
ON incidents FOR SELECT
USING (status = 'published');- Set environment variables:
NEXT_PUBLIC_SUPABASE_URL=https://xxx.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...- Configure the adapter:
{
id: 'database',
name: 'Production Database',
adapter: 'supabase',
adapterConfig: {
// Uses env vars by default, or override here
// url: 'https://your-project.supabase.co',
// anonKey: 'your-anon-key',
},
enabled: true,
}Row Level Security
Always enable RLS to control data access:
-- Only show published incidents publicly
CREATE POLICY "Public can view published"
ON incidents FOR SELECT
USING (status = 'published');
-- Authenticated users can view all
CREATE POLICY "Authenticated view all"
ON incidents FOR SELECT
TO authenticated
USING (true);API Adapter
Fetch incidents from any REST API that returns data in the expected format.
API Requirements
Your API endpoint should:
- Accept GET requests
- Return JSON with an
incidentsarray - Support CORS if hosted on a different domain
Configuration
{
id: 'external-api',
name: 'External Data',
adapter: 'api',
adapterConfig: {
endpoint: 'https://api.example.com/incidents',
// Optional: API key authentication
apiKey: process.env.API_KEY,
// Optional: custom headers
headers: {
'X-Custom-Header': 'value',
},
},
enabled: true,
}Building an API
If building your own API, return data in this format:
{
"incidents": [
{
"id": "...",
"temporal": { "date": "2024-01-15", "dateCertainty": "exact" },
"location": { /* ... */ },
// ... other fields
}
],
"meta": {
"total": 150,
"page": 1,
"pageSize": 100
}
}Multiple Sources
Combine multiple sources to aggregate data from different providers:
dataSources: [
{
id: 'official',
name: 'Official Reports',
color: '#3b82f6', // Blue
adapter: 'supabase',
adapterConfig: {},
enabled: true,
},
{
id: 'community',
name: 'Community Reports',
color: '#10b981', // Green
adapter: 'api',
adapterConfig: { endpoint: 'https://community-api.example.com' },
enabled: true,
},
{
id: 'historical',
name: 'Historical Archive',
color: '#8b5cf6', // Purple
adapter: 'static',
adapterConfig: { path: '/data/historical.json' },
enabled: false, // Off by default
},
]Each source gets a distinct marker color, and users can toggle them independently.
Creating Custom Adapters
For specialized data sources, you can create custom adapters.
Implementation
import type { DataAdapter, Incident, DataSourceConfig } from './types';
export class MyAdapter implements DataAdapter {
readonly id: string;
readonly name: string;
constructor(config: DataSourceConfig) {
this.id = config.id;
this.name = config.name;
}
async getIncidents(): Promise<Incident[]> {
// Fetch and transform your data
const response = await fetch('...');
const data = await response.json();
return this.transformData(data);
}
async getIncidentById(id: string): Promise<Incident | null> {
const incidents = await this.getIncidents();
return incidents.find(i => i.id === id) || null;
}
async getFilterOptions() {
const incidents = await this.getIncidents();
return {
countries: [...new Set(incidents.map(i => i.location?.country))],
siteTypes: [...new Set(incidents.map(i => i.location?.siteType))],
// ... other options
};
}
private transformData(raw: any[]): Incident[] {
return raw.map(item => ({
id: item.id,
status: 'published',
temporal: { date: item.date, dateCertainty: 'exact' },
location: {
name: item.location,
country: item.country,
latitude: item.lat,
longitude: item.lng,
siteType: 'unknown',
locationSensitivity: 'standard',
},
// ... map other fields
}));
}
}Registration
Register your adapter in src/adapters/index.ts:
import { MyAdapter } from './my-adapter';
export function createAdapter(config: DataSourceConfig): DataAdapter {
switch (config.adapter) {
case 'my-adapter':
return new MyAdapter(config);
// ... other cases
}
}Performance Tips
Large Datasets
For datasets over 1000 incidents:
- Use Mapbox clustering mode
- Implement server-side pagination
- Consider viewport-based loading
Caching
The data context caches fetched data. To refresh:
const { refresh } = useData();
await refresh();Error Handling
All adapters should handle errors gracefully:
async getIncidents(): Promise<Incident[]> {
try {
const response = await fetch(this.endpoint);
if (!response.ok) throw new Error('Fetch failed');
return await response.json();
} catch (error) {
console.error('Failed to fetch incidents:', error);
return []; // Return empty array on error
}
}