Source-Level Dependency Resolution

The Firefly compiler represents a fundamental shift in how F# code gets compiled to native executables. Unlike traditional F# compilation that relies on pre-compiled assemblies and the .NET runtime, Firefly compiles F# directly to native code through MLIR and LLVM, creating truly dependency-free executables. This architectural choice creates an interesting challenge: how do we handle library dependencies when we can’t rely on traditional assembly resolution?

Beyond Assembly-Based Dependencies

Traditional F# development uses a well-established pattern where open statements resolve to compiled assemblies in NuGet packages. When you write open System.Collections.Generic, the F# compiler knows exactly where to find the pre-compiled code. This works for .NET applications but breaks down completely when your goal is zero-dependency native executables.

In the Fidelity framework ecosystem, our core library Alloy provides essential functionality like memory management, numeric operations, and platform abstractions. However, Alloy exists as F# source code in your project’s lib directory, not as a compiled assembly. When a developer writes open Alloy.Memory in their Firefly application, the compiler needs to discover, parse, and include the relevant source files from the library ecosystem.

Consider this simple example that illustrates the complexity:

open Alloy.Memory
open Alloy.IO
open Alloy.IO.Console

let hello() =
    use buffer = stackBuffer<byte> 256
    prompt "Enter your name: "
    
    let name = 
        match readInto buffer with
        | Ok length -> spanToString (buffer.AsSpan(0, length))
        | Error _ -> "Unknown Person"
    
    name
    |> String.format "Hello, %s!"
    |> writeLine

[<EntryPoint>]
let main argv = 
    hello()
    0

This innocent-looking program actually depends on a range of functions across multiple Alloy modules, each with their own internal dependencies and platform-specific implementations. The Firefly compiler must discover all these dependencies, parse only what’s needed, and weave them together into a coherent compilation unit.

Farscape for C/C++ binding

A key aspect of our dependency resolution strategy that requires special attention is how we handle C/C++ libraries. We use a sophisticated header processing system that integrates with our overall dependency resolution.

Header and object files play a crucial role for static binding scenarios where C/C++ source code. The library targeted for static inclusion is compiled separately for object linking in the final stages of the LLVM process. Our approach maintains type safety and memory layout consistency across language boundaries.

Farscape: Automated Binding Generation

At the top level of our architecture, we leverage the Farscape CLI tool to process headers. Farscape serves two critical functions:

API Binding Generation: It creates idiomatic F# interfaces to C/C++ libraries by analyzing header files and generating appropriate type mappings, function declarations, and safety wrappers.
Memory Layout Mapping: Crucially, it works with BAREWire (our binary serialization and IPC library) to ensure consistent memory layouts between F# and C/C++ structures. This enables zero-copy operations and efficient inter-process communication.

When Farscape processes a header file, it generates a complete F# binding library with idiomatic functional wrappers for the target library’s available API. This provides the dependency resolution system with critical information about:

External library requirements
Platform-specific implementations
Memory layout constraints
Symbol mappings for LLVM integration

These generated libraries become first-class citizens in the Fidelity ecosystem, discoverable and usable through the same open statement mechanism as standard F# libraries.

The Three-Layer Solution

After extensive analysis, we’ve developed a hybrid approach to solving dependency inclusions that leverages the strengths of multiple parsing approaches working in concert. The solution operates on three distinct layers, each handling different aspects of the dependency resolution challenge.

Layer 1: File System Discovery

In our “earliest days” of framework development, we were simply using F# interactive to declare and include F# files directly into the sample program.

module Examples.HelloWorld

#if INTERACTIVE
#I ".\\lib\\Alloy"
#load "Console.fs"
#load "Memory.fs"
#load "IO.fs"
#endif

open Alloy.Memory
open Alloy.IO
open Alloy.IO.Console

let hello() =
    use buffer = stackBuffer<byte> 256
    prompt "Enter your name: "
    
    let name = 
        match readInto buffer with
        | Ok length -> spanToString (buffer.AsSpan(0, length))
        | Error _ -> "Unknown Person"
    
    name
    |> String.format "Hello, %s!"
    |> writeLine

[<EntryPoint>]
let main argv = 
    hello()
    0

We used “HelloWorld.fsx” (the ‘x’ file extension) directly for F# Compiler services to “pick up” the local files, but after that the compilation process is straightforward.

Layer 2: Metadata Parsing

Once we know what libraries exist, we need to understand their structure, dependencies, and platform-specific requirements. This is where XParsec proves invaluable, parsing structured metadata files that describe each library:

[package]
name = "OpenSSL-Bindings"
version = "1.0.0"
type = "c-binding"

[dependencies]
"Alloy.Memory" = ">=1.0.0"
"Alloy.Platform" = ">=1.0.0"

[targets]
windows = ["Windows.fs", "OpenSSL-Win.fs"]
linux = ["Linux.fs", "OpenSSL-Unix.fs"] 
embedded = ["Embedded.fs", "OpenSSL-Minimal.fs"]

[bindings]
library = "libssl"
headers = ["openssl/ssl.h", "openssl/crypto.h"]
header_processing = "farscape"
static_linking = true

For libraries generated by Farscape, the metadata includes additional sections that inform LLVM about required header files, linking strategies, and memory layout constraints. The XParsec parser builds a comprehensive dependency graph that accounts for all these factors.

Layer 3: Source Code Parsing

The final layer leverages F# Compiler Services (FCS) to parse all of the F# source files. This is critical because we need full semantic analysis to understand symbol usage, type relationships, and SRTP constraints. FCS provides robust, battle-tested parsing that maintains compatibility with standard F# syntax while giving us the AST representation needed for code path analysis and subsequent transformations.

let parseLibrarySourceFiles (sourceFiles: string list) : CompilerResult<ParsedSources> =
    let checker = FSharpChecker.Create()
    let parsingOptions = FSharpParsingOptions.Default
    
    sourceFiles
    |> List.map (fun file ->
        let sourceText = SourceText.ofString (File.ReadAllText(file))
        checker.ParseFile(file, sourceText, parsingOptions) 
        |> Async.RunSynchronously)
    |> combineParseResults

The Developer Experience: Transparent Dependency Resolution

A critical aspect of our approach is keeping the complexity at a minimum from developers. While the architecture is sophisticated, the developer experience aspires to remain straightforward. Fidelity seeks to achieve this through a blend of editor integration and “native” compiler intelligence. (pun!)

Shadow Project Generation

The key point of exploration in our design is the concept of “shadow scaffold.” When a developer opens a Fidelity framework project, a planned language services extension creates temporary shadow files that FCS can understand:

// Example of a generated shadow file (never visible to the developer)
#I ".\\lib\\Alloy"
#I ".\\lib\\BAREWire"
#I ".\\lib\\Farscape-Output\\OpenSSL-Bindings"
#load "Memory.fs"
#load "Console.fs"
#load "OpenSSL-Win.fs"
#load "Networking.fs" 
#load "C:\\Users\\Developer\\Projects\\MyProject\\src\\main.fs"

These shadow files act as a bridge between our TOML-based dependency system and traditional F# tooling. By leveraging the #load directive that F# Interactive understands, we can provide full IntelliSense and compiler feedback while maintaining our source-based approach.

Integrated Development Environment

We’re currently working on a Fidelity extension for Visual Studio Code that aims to bridge the gap between traditional F# development and our native compilation approach. When working with Fidelity projects, the extension:

Monitors changes to both source files and .fidproj files
Generates shadow files based on the dependency graph
Will Direct FCS to use these shadow files for IntelliSense and error checking
Provide LSP and highlighting support for all intermediate representations (.fcs, .oak, .mlir, .ll)

The extension would aspire to integrate with existing F# editor tools like Ionide. This may require some work in those “stacks” which we plan to contribute, but the end goal is a hybrid environment that supports Fidelity framework projects to the extent they can be made to look and feel like idiomatic F# workflow.

# Example .fidproj file that the extension understands
[package]
name = "my_embedded_app"
version = "0.1.0"
authors = ["Developer <dev@example.com>"]

[dependencies]
alloy = "0.5.0"
fsil = "1.0.0"
openssl-bindings = { version = "0.3.0", features = ["minimal"] }

[platform]
default = "stm32l5"

When a developer opens this project, the Fidelity extension would discover and load all required dependencies, making them available for IntelliSense and compilation without any manual configuration.

Practical Workflow: From Development to Deployment

The integrated workflow would enable developers to move from concept to deployment with minimal friction:

Create a new Fidelity project with fargo new my_project
Edit F# source files with full IntelliSense support
Resolve dependencies automatically through the Fidelity VSCode extension
Compile to native code with fpm build --platform stm32l5 and/or VSCode tooling
Deploy the resulting binary directly to the target platform

Throughout this process, the developer would rarely need to manually specify file paths, manage include directives, or configure compiler settings. There may be some early exceptions to this if the developer is building out their own dependency hierarchy and may have a mix of local and remote package includes. But the framework has designs to manage all this complexity in a clear and consistent manner, allowing the developer to quickly turn to considering the goals of their F# code.

LLVM Integration for Native Binding

A key advantage of our approach is the seamless integration with LLVM for static binding to C/C++ libraries. When the library is included it should provide the necessary “hooks” to access the local library source. From there the necessary meta information is extracted from that library and passed into the compilation pipeline in order to support static linking where configured by the developer.

let extractHeadersForLLVM (library: LibraryMetadata) : HeaderInfo list =
    match library.Bindings with
    | Some bindings when bindings.StaticLinking ->
        bindings.Headers 
        |> List.map (fun headerPath -> 
            { Path = headerPath
              ProcessingMode = bindings.HeaderProcessing
              IncludePaths = bindings.IncludePaths })
    | _ -> []

This allows LLVM to understand the complete type information and memory layouts defined in the C/C++ headers. The result is type-safe, memory-safe handling of those native bindings with optimal performance characteristics.

Intelligent Tree-Shaking

The power of this hybrid approach becomes apparent when we consider tree-shaking. Traditional F# compilation includes entire assemblies, even when you only use a single function. Our source-level approach enables surgical precision in determining exactly what code needs to be compiled.

The tree-shaking process operates through reachability analysis on the Oak AST (a simplified, canonical representation of F# code). Starting from entry points like [<EntryPoint>] functions, we trace through symbol usage to build a minimal dependency set:

let performReachabilityAnalysis (entryPoints: string list) (dependencyGraph: Map<string, string list>) : Set<string> =
    let rec trace visited remaining =
        match remaining with
        | [] -> visited
        | current :: rest ->
            if Set.contains current visited then
                trace visited rest
            else
                let newVisited = Set.add current visited
                let dependencies = Map.tryFind current dependencyGraph |> Option.defaultValue []
                let newRemaining = dependencies @ rest
                trace newVisited newRemaining
    
    trace Set.empty entryPoints

For our HelloWorld example, this analysis might discover that we only need:

stackbuffer from Alloy.Memory
prompt and readInto from Alloy.IO
A handful of supporting functions and type definitions

This intelligent tree-shaking applies equally to Farscape-generated bindings, ensuring that only the C/C++ functions actually used in the program are included in the final executable. The embodies Bjarne Stroustrup’s famous quote, “only pay for what you use”.

BAREWire Integration: Memory Mapping Across Boundaries

One of the most exciting aspects of our architecture design is the integration with BAREWire, our library for binary serialization, memory mapping, and IPC. When Farscape processes headers, it generates not only F# bindings for the C/C++ API but also BAREWire serialization code for all relevant structures.

This enables several critical capabilities:

Zero-copy operations between F# and C/C++ code
Efficient IPC between processes built with Fidelity
Consistent memory layouts across language boundaries
Safe serialization of complex data structures

BAREWire’s integration with Farscape-generated bindings creates a seamless experience for developers, who can write normal F# code without worrying about the complexities of memory mapping and native interop.

Platform-Aware Compilation

One of the most sophisticated aspects of our dependency resolution is platform awareness. Modern software must handle diverse deployment targets, from Windows desktop applications to embedded ARM controllers. Our metadata-driven approach allows libraries to specify platform-specific implementations while maintaining a unified API surface.

When compiling for an embedded target, the OpenSSL bindings might include only cryptographic primitives, while a desktop compilation could include the full SSL/TLS stack. The dependency resolver automatically selects the appropriate source files based on the target platform specified in the compilation command.

The Compilation Pipeline Integration

This dependency resolution system integrates seamlessly with Firefly’s existing compilation pipeline. The enhanced translation process follows these steps:

Parse the main source file to extract open statements
Resolve metadata using XParsec to build dependency graphs
Discover available libraries through file system scanning and frgo.dev queries
Parse all required source files using FCS
Merge ASTs into a unified Oak representation
Tree-shake to remove unused code paths
Transform through closure elimination and union layout optimization
Generate MLIR and relevant passes for transforms into LLVM
Transform LLVM static binding and link-time optimizations (LTO)
Compile using LLVM to produce the binary for a given platform target

Each step will maintain diagnostic information, ensuring that compilation errors provide clear guidance about which library and source file contains the problematic code.

The Language Services Architecture

The editor integration that powers our developer experience follows a layered architecture that provides incremental responsiveness and precise feedback:

1. Project Monitoring

The Fidelity extension monitors both source files and .fidproj files, triggering appropriate actions when dependencies change:

// Conceptual representation from the extension
function watchProjectFiles() {
    const watcher = vscode.workspace.createFileSystemWatcher("**/*.fidproj");
    watcher.onDidChange(async uri => {
        await refreshDependencies(uri);
        await regenerateShadowFiles(uri);
        await notifyLanguageServer(uri);
    });
}

2. Shadow File Generation

When dependencies change, the extension generates temporary shadow files that serve as a bridge between our metadata-based system and F# Compiler Services:

let generateShadowFile (sourceFile: string) (dependencies: ProjectDependencies) =
    let tempDir = Path.Combine(Path.GetTempPath(), "Fidelity", "Shadow")
    Directory.CreateDirectory(tempDir) |> ignore
    
    let shadowPath = Path.Combine(tempDir, Path.GetFileName(sourceFile) + ".shadow.fsx")
    
    use writer = new StreamWriter(shadowPath)
    
    // Write library paths
    for lib in dependencies.LibraryPaths do
        writer.WriteLine($"#I \"{lib}\"")
    
    // Write required #load directives 
    for file in dependencies.SourceFiles do
        writer.WriteLine($"#load \"{file}\"")
    
    // Load the actual source file
    writer.WriteLine($"#load \"{Path.GetFullPath(sourceFile)}\"")
    
    shadowPath

3. FCS Integration

In the future our extended language service will direct F# Compiler Services to parse and analyze these shadow files, providing IntelliSense and error checking:

// Inside the language service
let parseShadowFile (shadowFile: string) =
    let checker = FSharpChecker.Create()
    let source = File.ReadAllText(shadowFile)
    let sourceText = SourceText.ofString source
    
    // Parse with FCS - this will follow all the #load directives
    checker.ParseFile(shadowFile, sourceText, parsingOptions = FSharpParsingOptions.Default) 
    |> Async.RunSynchronously

This architecture ensures that developers receive the same high-quality editor experience they expect from traditional F# development, while enabling the advanced capabilities of our source-based native compilation approach.

Technical Implementation Notes

For developers interested in contributing to or extending this system, several key implementation details are worth understanding:

Incremental Compilation: The dependency graph would enable intelligent incremental compilation in the future. When a single source file changes, our goal is that only affected dependencies need recompilation, dramatically improving development cycle times.

Error Propagation: Diagnostic information should flow through all layers, ensuring that compilation errors reference the original source locations in library files, not internal compiler representations.

Extensibility: The metadata format will be extensible to support new library types, compilation flags, and platform targets without affecting existing tooling.

Looking to the Future: Fargo

While our current implementation focuses on local library resolution, we’re actively developing a comprehensive package management system called Fargo that will extend these concepts to a full-fledged ecosystem.

Inspired by Rust’s Cargo system, Fargo will provide a streamlined way to discover, distribute, and consume Fidelity packages. The package manager will be available via both fargo and fpm (Fidelity Package Manager) commands, offering a familiar experience for developers:

# Create new project
fargo new my_project

# Add dependencies
fpm add alloy
fpm add fsil

# Build for a specific platform
fpm build --platform stm32l5

Projects will use the .fidproj format – a TOML-based project definition that combines the best aspects of traditional F# projects with the clarity and flexibility of modern package management:

[package]
name = "my_project"
version = "0.1.0"
authors = ["Author Name <email@example.com>"]

[dependencies]
alloy = "0.5.0"
fsil = "1.0.0"
fidelity_stm32 = { version = "0.3.0", features = ["l5"] }

[platform]
default = "stm32l5"

Packages will be distributed as .fidpkg files – compressed archives containing source code, metadata, and any required native components. The registry system will support both centralized and decentralized models, with federation capabilities for enterprise and air-gapped environments.

The integration between Fargo, Farscape, and Firefly will create a seamless development experience from package discovery to deployment:

Discover libraries through the Fargo registry
Process C/C++ headers automatically with Farscape to create your own F# binding library
Add dependencies to your project
Build your application with Firefly
Deploy your applicaton to your target platform

This ecosystem approach will enable the Fidelity Framework to grow organically while maintaining the core principles of zero-cost abstractions, type safety, and platform adaptability. By building on the foundation of source-level dependency resolution described in this article, we’re creating a development environment that combines the best aspects of traditional F# with the performance and flexibility of native compilation.

As we continue to expand the Fidelity ecosystem, we invite the community to observe our process in building the future of F# native development. Whether you’re developing embedded firmware, high-performance accelerated applications, or anything in between, the Fidelity Framework aims to provide the tools and libraries you need to write expressive, safe, and efficient code for any platform.

Return to Blog