James Arthur’s talk on local-first software development with Elixir, expounds on an async engine known as Electric SQL and its integration with web frameworks like Phoenix. James outlines the benefits of local-first applications, which utilize embedded local data stores to enhance user experience, particularly in scenarios with poor connectivity, and introduces Electric as a solution for automating data synchronization without the typical pitfalls of client-server models. He details the architecture of Electric, highlighting its use of standard HTTP for scalability, and explains how it operates in tandem with tools like PostgreSQL and CDN infrastructures to efficiently handle data delivery. Demonstrating practical examples, including a robust to-do list app, James delves into how authorization is managed, the implementation of optimistic state patterns for offline resiliency, and the overarching design principles that facilitate seamless syncing of mutable data. Throughout the presentation, he reflects on the evolution of their company towards local-first solutions, emphasizing the importance of existing libraries and potential for standardized sync protocols in future developments.
Local-first software development with Elixir: A comprehensive overview
What is local-first software development?
Local-first software prioritizes storing data locally on the user’s device rather than constantly relying on server-side databases. Instead of making synchronous API calls for every data request, applications work seamlessly offline and sync in the background when connectivity is available.
James positioned Electric SQL as a sync engine that enables local-first applications using PostgreSQL, mobile, and web platforms. Unlike the traditional client-server model that often leads to latency issues and dependency on network availability, local-first software:
- Enhances user experience by making interactions instantaneous.
- Supports offline functionality without requiring complex caching mechanisms.
- Improves scalability by reducing the burden on backend servers.
This approach is particularly well-suited for Elixir-based applications due to Elixir’s strengths in concurrency, real-time processing, and fault tolerance.
How Electric SQL powers local-first applications
James explained that Electric SQL acts as an Elixir-based middleware sitting in front of PostgreSQL. It works by:
- Establishing a logical replication subscription to track database changes.
- Providing partial replication through a concept called shapes, which allows clients to subscribe to specific data subsets.
- Syncing data efficiently using long polling instead of WebSockets for better scalability.
Unlike real-time sync solutions that rely on constant database queries, Electric leverages shapes and CDNs like Cloudflare and Fastly to serve static JSON documents, reducing database load and improving response times.
Performance and scalability
One of the most compelling aspects of Electric SQL is its scalability. James highlighted that the system can handle over a million concurrent clients while maintaining stable memory usage and linear sync performance. The Erlang VM, which powers Elixir, plays a crucial role in ensuring resilience under heavy loads.
Integrating local-first syncing in Elixir and Phoenix
James walked through a practical implementation of Electric SQL with Elixir and Phoenix LiveView, demonstrating how:
- Data synchronization works in real-time, updating UI elements instantly without explicit JavaScript calls.
- Phoenix Sync and Ecto queries seamlessly expose synchronized data to the frontend.
- Authorization mechanisms ensure that only authenticated users access synced data, integrating with Phoenix’s plug-based middleware.
A to-do list app served as a compelling example, showing how multiple users could edit tasks across different browsers without conflicts. James also introduced gatekeeper endpoints that generate authentication tokens for secure access to the streaming API.
Handling offline scenarios and optimistic updates
A key challenge of local-first software is ensuring data consistency when users interact with the application while offline. Jame introduced the optimistic state pattern, which allows changes to be made locally before the server confirms them.
Optimistic State with React’s useOptimistic Hook
For frontend applications, Electric integrates with React, enabling developers to use useOptimistic to:
- Provide instant feedback when users modify data.
- Reduce the perceived lag of syncing operations.
Persistent Local Storage with Velo
For more robust offline handling, James introduced Velo, a reactive local store that shares the optimistic state across components and persists data in local storage. This approach minimizes data conflicts when multiple users modify the same record.
How Electric SQL handles data writes
One of the standout features of Electric SQL is its ability to batch and queue write operations. James explained that:
- Changes are first written to an embedded PostgreSQL database on the client’s machine.
- A shadow table tracks modifications before syncing them to the main database.
- When connectivity is restored, batched transactions are sent to the backend for processing.
This ensures that applications remain responsive and functional even under poor network conditions.
The future of local-first software development
James wrapped up the talk by reflecting on the evolution of Electric SQL. Originally focused on distributed databases, the company pivoted to local-first solutions after recognizing the growing demand for offline-capable applications.
The road ahead
He also mentioned exciting developments in the local-first ecosystem, such as:
- Zero Sync, Jazz, and Instant DB — alternative approaches to local-first data synchronization.
- Standardized Sync Protocols — a future possibility for unifying local-first software approaches across platforms.
The takeaway? Developers should focus on leveraging existing sync engines rather than reinventing the wheel.
Final thoughts: Why local-first with Elixir matters
James Arthur’s talk showcased how Electric SQL and Elixir provide a scalable, efficient, and developer-friendly way to implement local-first applications.
Why this matters:
- Local-first software enhances performance and resilience.
- Electric SQL simplifies syncing, making local-first development accessible.
- Elixir’s fault tolerance and concurrency make it the perfect match for this architecture.
For developers frustrated by latency issues, network dependencies, or complex sync logic, Electric SQL offers a compelling alternative. As the demand for real-time, offline-capable applications grows, Elixir and Phoenix may become a dominant force in shaping the future of local-first development.