James Trunk, VP of engineering at Griffin, discusses how his team utilizes Clojure’s superpowers, specifically Tap and Portal, to understand and debug their event-driven architecture in building Griffin, a banking-as-a-service platform. Griffin simplifies the financial ecosystem for fintechs by offering a platform with a well-documented API and extensive use of Clojure. The architecture includes transducers, functions that describe a transition from one state to another, which are crucial for the smooth integration of new components and the replacement of Foundation DB without affecting the processes. Portal, a debugging tool in Clojure, is introduced as a powerful window into the system, providing real-time data inspection and context-aware views, enabling efficient debugging of complex data structures.
Tapping into one of Clojure’s superpowers with Portal: A comprehensive overview
An introduction to Griffin’s architecture
James Trunk begins by setting the stage with a brief overview of Griffin, a banking-as-a-service platform designed to streamline financial processes for fintech companies. The platform aims to simplify the integration of financial services, offering features like lending through a well-documented API. The system’s architecture, built extensively using Clojure, includes key components such as:
- Customer requests to API: The entry point for users to interact with the platform.
- Authorization and authentication checks: Ensuring secure access.
- Foundation DB (FDB): Serving as an event source queue.
- Procs: Small processes that listen for specific messages and process them.
This decoupled architecture not only scales efficiently but also hides the complexities of third-party APIs, providing a seamless user experience.
The role of transducers in Clojure
A significant highlight of the presentation is the use of transducers in Griffin’s system. Transducers, a functional programming concept, are composable functions that describe state transitions. They are agnostic to both the input and output state, allowing for flexible and efficient data processing. James uses metaphors to explain how their system, akin to a body’s nervous system, smoothly transitioned its Foundation DB without disrupting the core processes, thanks to the power of transducers.
Transducers enable the separation of the recipe (logic) from the ingredients (data), making them a powerful tool for building scalable and maintainable systems.
Procs: Composable and efficient building blocks
James elaborates on the benefits of using procs in their architecture. These composable units are not only flexible but also highly efficient due to their eager implementation and lack of intermediate state. He presents a proc used for making deposits, highlighting its data-oriented and declarative nature. This approach ensures that the system remains modular and easy to test.
Introducing Portal: Enhancing debugging in Clojure
One of the key superpowers in Clojure that James emphasizes is Portal, a tool that significantly enhances debugging capabilities. Portal integrates with Visual Studio Code, allowing developers to inspect data in real-time. Unlike traditional debugging methods that rely on print statements or logs, Portal offers a more data-oriented approach by utilizing Clojure’s tap> function to send data directly to the debugging tool.
Portal provides a rich, context-aware view of data, supporting various views such as tree and table views. This feature enables developers to explore nested data structures deeply and filter messages based on specific criteria, making it easier to navigate and understand complex data.
Practical demonstrations and insights
During the presentation, James demonstrates Portal’s ability to detect sequences of maps and convert them into tables for easier viewing. He also showcases how to visualize finite state machines and their transitions, and perform diffs on multiple messages to identify differences. These demonstrations highlight the power and flexibility of using Clojure’s transducers, tap, and Portal in a real-world debugging scenario.
The transition from Kafka to Foundation DB
In the Q&A session, James addresses why Griffin shifted from using Kafka to Foundation DB. The primary reasons were performance-related; Kafka, while robust, introduced too many network latencies, making the system slower. Foundation DB, on the other hand, provided similar reliability and scalability but with better performance and transactional integrity. This shift allowed Griffin to maintain their proc-based architecture without making the processes monolithic.
Conclusion
James Trunk’s presentation offers valuable insights into the advanced capabilities of Clojure, particularly with tools like Tap and Portal. These tools not only simplify debugging but also enhance the efficiency and flexibility of event-driven architectures. As fintech companies like Griffin continue to innovate, leveraging the superpowers of Clojure can provide a significant competitive advantage, ensuring robust, scalable, and maintainable systems. For developers interested in functional programming and advanced debugging techniques, exploring Clojure’s features like transducers and Portal is highly recommended.