This documentation is also published as Markdown for efficient machine reading: the whole site is indexed at /llms.txt, and every page has a clean Markdown copy under /_llms/. These are generated from the same source and cost far fewer tokens to read than this rendered HTML.

Skip to main content Skip to navigation
Guides

Source content from outside the markdown pipeline

Implement IContentService to surface JSON files, a database table, or a remote API as routed pages, navigation entries, search documents, and xref targets.

Implement IContentService directly to give content the markdown pipeline can't reach — a folder of JSON release notes, a SQL table, a remote API, generated API reference — its own routed pages, navigation entries, cross-references, and search documents, exactly the way markdown pages get them.

Two narrower needs have shorter answers. For a second markdown tree with a different front-matter type, use chained AddMarkdownContent<T> instead — see Serve docs and a blog from separate content roots. When a dataset feeds existing pages but needs no routes of its own, register it with AddDataFile<T> rather than a content service — see Use a YAML or JSON data file in pages.

The recipe references examples/ExtensibilityLabExample/ReleaseNotesContentService.cs, which turns Content/releases/*.json into /releases/{version}/ routes.

Before you begin

  • A working Pennington site on bare AddPennington (see Create your first Pennington site if not). AddDocSite pins its own markdown service, so adding a second content service on top works, but the concepts below assume the unwrapped host.
  • Familiarity with the four-stage pipeline at a conceptual level (The content pipeline and union types).
  • Source data that can be enumerated synchronously or asynchronously on startup — DiscoverAsync runs both at build time and on demand for live requests.

Write the service

Implement Pennington.Content.IContentService as a sealed class. Cache the parsed records in a Lazy<ImmutableList<T>> so discovery and the TOC share one pass over the source. That cache holds for the service's lifetime, which has consequences for live edits — see the stale-data warning under Register the service.

csharp 
namespace ExtensibilityLabExample;
  
using System.Collections.Immutable;
using System.Globalization;
using System.Text.Json;
using System.Text.Json.Serialization;
using Microsoft.AspNetCore.Hosting;
using Pennington.Content;
using Pennington.FrontMatter;
using Pennington.Pipeline;
using Pennington.Routing;
using Pennington.Search;
using Pennington.StructuredData;
  
/// <summary>
/// Demonstrates <see cref="IContentService"/> by turning a folder of
/// <c>Content/releases/*.json</c> files into site pages, a navigation
/// section, and cross-reference entries.
/// <para>
/// Emits one <see cref="DiscoveredItem"/> per JSON file plus an index
/// route. The static-build crawler fetches each one over HTTP from the
/// running host, so a sibling <c>MapGet("/releases/{version}/")</c>
/// endpoint in <c>Program.cs</c> does the actual HTML rendering — the
/// same code path dev-mode uses. That keeps the service focused on
/// discovery, TOC, and cross-references and leaves presentation to the
/// endpoint.
/// </para>
/// <para>
/// Backs how-to 2.3.10 <c>/how-to/extensibility/custom-content-service</c>.
/// </para>
/// </summary>
public sealed class ReleaseNotesContentService : IContentService
{
    private readonly string _releasesDirectory;
    private readonly Lazy<ImmutableList<ReleaseEntry>> _entriesLazy;
  
    public ReleaseNotesContentService(IWebHostEnvironment environment)
    {
        _releasesDirectory = Path.Combine(environment.ContentRootPath, "Content", "releases");
        _entriesLazy = new Lazy<ImmutableList<ReleaseEntry>>(LoadEntries);
    }
  
    public string DefaultSectionLabel => "Releases";
    public int SearchPriority => 20;
  
    /// <summary>The full set of release entries this service knows about.</summary>
    public IReadOnlyList<ReleaseEntry> Entries => _entriesLazy.Value;
  
    /// <summary>Find a single release by version string, or null if no match.</summary>
    public ReleaseEntry? TryGet(string version) =>
        _entriesLazy.Value.FirstOrDefault(e => e.Version == version);
  
    /// <summary>
    /// One discovered item for the index plus one per JSON file. Each route is
    /// paired with <see cref="EndpointSource"/> — the build crawler discovers
    /// the URL and fetches it through the live pipeline, where the sibling
    /// <c>MapGet</c> endpoint in <c>Program.cs</c> produces the HTML. Because the
    /// endpoint serves real canonical HTML, these routes are included in
    /// <c>sitemap.xml</c> like any other page.
    /// <para>
    /// Each release item carries its <see cref="ReleaseEntry"/> as
    /// <see cref="DiscoveredItem.Metadata"/>. That single assignment surfaces the records to
    /// discovery: the default <c>GetRecordsAsync</c> bridge picks them up, so the browse-by-channel
    /// taxonomy, the custom <c>channel</c> search facet, and the per-page JSON-LD all light up from
    /// the one record — no separate index page, the same treatment markdown gets.
    /// The index item carries no metadata, so it is not itself a record.
    /// </para>
    /// </summary>
    public async IAsyncEnumerable<DiscoveredItem> DiscoverAsync()
    {
        yield return new DiscoveredItem(
            ContentRouteFactory.FromUrl(new UrlPath("/releases/")),
            new EndpointSource());
  
        foreach (var entry in _entriesLazy.Value)
        {
            var route = ContentRouteFactory.FromCustom(
                url: new UrlPath($"/releases/{entry.Version}/"),
                sourceFile: new FilePath(entry.SourcePath));
            yield return new DiscoveredItem(route, new EndpointSource()) { Metadata = entry };
        }
  
        await Task.CompletedTask;
    }
  
    /// <summary>No static files to copy — JSON sources are transformed, not republished.</summary>
    public Task<ImmutableList<ContentToCopy>> GetContentToCopyAsync()
        => Task.FromResult(ImmutableList<ContentToCopy>.Empty);
  
    /// <summary>TOC entries so the pages show up in navigation and the search index.</summary>
    public Task<ImmutableList<ContentTocItem>> GetContentTocEntriesAsync()
    {
        var entries = _entriesLazy.Value;
        var builder = ImmutableList.CreateBuilder<ContentTocItem>();
  
        builder.Add(new ContentTocItem(
            Title: "Releases",
            Route: ContentRouteFactory.FromUrl(new UrlPath("/releases/")),
            Order: 100,
            HierarchyParts: ["releases"],
            SectionLabel: DefaultSectionLabel,
            Locale: null));
  
        var order = 110;
        foreach (var entry in entries)
        {
            builder.Add(new ContentTocItem(
                Title: entry.Title,
                Route: ContentRouteFactory.FromUrl(new UrlPath($"/releases/{entry.Version}/")),
                Order: order,
                HierarchyParts: ["releases", entry.Version],
                SectionLabel: DefaultSectionLabel,
                Locale: null));
            order += 10;
        }
  
        return Task.FromResult(builder.ToImmutable());
    }
  
    /// <summary>One cross-reference per release so <c>&lt;xref:release-1.0.0&gt;</c> resolves.</summary>
    public Task<ImmutableList<CrossReference>> GetCrossReferencesAsync()
    {
        var entries = _entriesLazy.Value;
        var builder = ImmutableList.CreateBuilder<CrossReference>();
  
        foreach (var entry in entries)
        {
            var route = ContentRouteFactory.FromUrl(new UrlPath($"/releases/{entry.Version}/"));
            builder.Add(new CrossReference($"release-{entry.Version}", entry.Title, route));
        }
  
        return Task.FromResult(builder.ToImmutable());
    }
  
    private ImmutableList<ReleaseEntry> LoadEntries()
    {
        if (!Directory.Exists(_releasesDirectory))
        {
            return [];
        }
  
        var builder = ImmutableList.CreateBuilder<ReleaseEntry>();
        foreach (var file in Directory.EnumerateFiles(_releasesDirectory, "*.json"))
        {
            var json = File.ReadAllText(file);
            var dto = JsonSerializer.Deserialize<ReleaseJson>(json, JsonOptions);
            if (dto is null)
            {
                continue;
            }
  
            builder.Add(new ReleaseEntry
            {
                Version = dto.Version,
                Title = dto.Title,
                Date = DateTime.TryParse(dto.Date, CultureInfo.InvariantCulture,
                    DateTimeStyles.AssumeUniversal | DateTimeStyles.AdjustToUniversal, out var parsed)
                        ? parsed
                        : null,
                Channel = string.IsNullOrWhiteSpace(dto.Channel) ? "stable" : dto.Channel!,
                Tags = dto.Tags?.ToArray() ?? [],
                Highlights = dto.Highlights ?? [],
                SourcePath = file,
            });
        }
  
        return [.. builder.OrderBy(e => e.Version, StringComparer.OrdinalIgnoreCase)];
    }
  
    private static readonly JsonSerializerOptions JsonOptions = new()
    {
        PropertyNameCaseInsensitive = true,
    };
  
    private sealed record ReleaseJson(
        string Version,
        string Title,
        string Date,
        string? Channel,
        List<string>? Tags,
        List<string>? Highlights);
}
  
/// <summary>
/// One parsed release record. Implements <see cref="IFrontMatter"/> plus the discovery capability
/// mixins, so a single record feeds the browse-by-channel taxonomy (<see cref="ITaggable"/> /
/// the <c>Channel</c> key), the custom <c>channel</c> search facet (<see cref="IHasSearchFacets"/>),
/// and the per-page JSON-LD (<see cref="IHasStructuredData"/>) — the same treatment markdown front
/// matter gets, with no extra wiring beyond attaching it to the discovered item.
/// </summary>
public sealed record ReleaseEntry : IFrontMatter, ITaggable, IHasSearchFacets, IHasStructuredData
{
    /// <summary>Semantic version, used as the route slug (e.g. <c>1.1.0</c>).</summary>
    public string Version { get; init; } = "";
  
    /// <inheritdoc/>
    public string Title { get; init; } = "";
  
    /// <inheritdoc/>
    public DateTime? Date { get; init; }
  
    /// <summary>Release channel (<c>stable</c>, <c>beta</c>, ...) — the taxonomy key and custom search facet.</summary>
    public string Channel { get; init; } = "stable";
  
    /// <inheritdoc/>
    public string[] Tags { get; init; } = [];
  
    /// <summary>Bullet highlights rendered on the detail page.</summary>
    public IReadOnlyList<string> Highlights { get; init; } = [];
  
    /// <summary>Absolute path of the JSON source file (drives file-watching).</summary>
    public string SourcePath { get; init; } = "";
  
    /// <inheritdoc/>
    public IReadOnlyDictionary<string, string[]> SearchFacets => new Dictionary<string, string[]>
    {
        ["channel"] = [Channel],
    };
  
    /// <inheritdoc/>
    public IEnumerable<JsonLdEntity> GetStructuredData(StructuredDataContext context) =>
    [
        new ReleaseJsonLd
        {
            Name = Title,
            SoftwareVersion = Version,
            DatePublished = Date?.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture),
            Url = context.CanonicalUrl,
        },
    ];
}
  
/// <summary>A schema.org <c>SoftwareApplication</c> describing one release, emitted as JSON-LD.</summary>
public sealed record ReleaseJsonLd : JsonLdEntity
{
    /// <inheritdoc/>
    [JsonPropertyName("@type")]
    public override string Type => "SoftwareApplication";
  
    /// <summary>Release title.</summary>
    [JsonPropertyName("name")]
    public required string Name { get; init; }
  
    /// <summary>Semantic version string.</summary>
    [JsonPropertyName("softwareVersion")]
    public string? SoftwareVersion { get; init; }
  
    /// <summary>ISO publication date.</summary>
    [JsonPropertyName("datePublished")]
    public string? DatePublished { get; init; }
  
    /// <summary>Canonical URL of the release page.</summary>
    [JsonPropertyName("url")]
    public string? Url { get; init; }
}

Three non-obvious moves carry this service:

  • DiscoverAsync pairs a route with a ContentSource case. Build the route with ContentRouteFactory.FromUrl (synthetic URL, no backing file) or ContentRouteFactory.FromCustom (URL plus an on-disk FilePath so file-watching picks up edits). The example uses EndpointSource so the build crawler fetches each URL through a sibling MapGet endpoint; those routes serve real canonical HTML, so they appear in sitemap.xml like any other page.
  • GetContentTocEntriesAsync feeds the sidebar and the search index. Set Title, Route, Order (10/20/30 spacing), HierarchyParts (sidebar nesting), and SectionLabel (group header). The same items power search ranking.
  • GetCrossReferencesAsync publishes one CrossReference(uid, title, route) per record so authors can deep-link entries with <xref:uid>. Pick a stable prefix (release-1.0.0 here) so the uid does not depend on a URL that may move.

GetContentToCopyAsync returns ImmutableList.Empty when HTML served by an endpoint is the only output. Byte artifacts (a robots.txt, a JSON sidecar) are not a content-service concern — they belong on IArtifactContentService (Emit generated output artifacts).

Register the service

AddPennington does not auto-discover IContentService implementations — register directly on IServiceCollection. When an endpoint in Program.cs needs the concrete type to render detail pages, register it once by concrete type and forward IContentService to the same instance so the container does not create a second copy.

csharp 
builder.Services.AddSingleton<ReleaseNotesContentService>();
builder.Services.AddSingleton<IContentService>(sp =>
    sp.GetRequiredService<ReleaseNotesContentService>());

This service reads its JSON into a Lazy<T> once and lives as a singleton, so an edit to a release file during a dev session is not picked up until restart. When live-reload on source edits matters, register the service file-watched instead — AddFileWatched<T> plus a transient IContentService wrapper, as Publish a custom feed from a content service shows. AddSingleton<IContentService> over a file-watched type silently caches the first copy and serves stale data. To source from a remote API rather than disk, see Source content from a remote API.

Feed your records to taxonomy, search, and JSON-LD

The service above produces routes, navigation, and cross-references. To also let your records drive browse-by-field pages, custom search facets, and JSON-LD — the same way markdown records do — give each record a typed front matter that implements the capability mixins, and attach it to the discovered item. The example's ReleaseEntry does exactly this:

csharp 
public sealed record ReleaseEntry : IFrontMatter, ITaggable, IHasSearchFacets, IHasStructuredData
{
    /// <summary>Semantic version, used as the route slug (e.g. <c>1.1.0</c>).</summary>
    public string Version { get; init; } = "";
  
    /// <inheritdoc/>
    public string Title { get; init; } = "";
  
    /// <inheritdoc/>
    public DateTime? Date { get; init; }
  
    /// <summary>Release channel (<c>stable</c>, <c>beta</c>, ...) — the taxonomy key and custom search facet.</summary>
    public string Channel { get; init; } = "stable";
  
    /// <inheritdoc/>
    public string[] Tags { get; init; } = [];
  
    /// <summary>Bullet highlights rendered on the detail page.</summary>
    public IReadOnlyList<string> Highlights { get; init; } = [];
  
    /// <summary>Absolute path of the JSON source file (drives file-watching).</summary>
    public string SourcePath { get; init; } = "";
  
    /// <inheritdoc/>
    public IReadOnlyDictionary<string, string[]> SearchFacets => new Dictionary<string, string[]>
    {
        ["channel"] = [Channel],
    };
  
    /// <inheritdoc/>
    public IEnumerable<JsonLdEntity> GetStructuredData(StructuredDataContext context) =>
    [
        new ReleaseJsonLd
        {
            Name = Title,
            SoftwareVersion = Version,
            DatePublished = Date?.ToString("yyyy-MM-dd", CultureInfo.InvariantCulture),
            Url = context.CanonicalUrl,
        },
    ];
}
csharp 
yield return new DiscoveredItem(route, new EndpointSource()) { Metadata = entry };

That Metadata assignment lets taxonomy, search, and JSON-LD read the record: the engine reads it through GetRecordsAsync (the default bridges from DiscoverAsync, so attaching metadata is all it takes — no override needed):

  • Taxonomy — AddTaxonomy<ReleaseEntry, string>(opts => opts.SelectKey = fm => fm.Channel) gives you /channel/ browse pages with no FileSource required (see Build browse-by-{field} pages with AddTaxonomy).
  • Search facets — the IHasSearchFacets channel axis emits alongside the built-in section/tag/area dimensions.
  • JSON-LD — the IHasStructuredData entity is injected into each release page's <head> automatically when CanonicalBaseUrl is set; no <script> to hand-write.

A record participates in a taxonomy axis only when its metadata is that axis's TFrontMatter, so type your records as the front matter you intend to browse. If you project records that don't flow through DiscoverAsync (or want to filter which ones do), override GetRecordsAsync directly instead of attaching Metadata.

Result

The discovered records produce a "Releases" section in the sidebar, one route per entry, and one xref id per entry:

text 
/releases/                  -> Releases (index)
/releases/1.0.0/            -> uid: release-1.0.0
/releases/1.1.0/            -> uid: release-1.1.0

Each /releases/{version}/ URL renders through the sibling MapGet endpoint, the entries appear in the search index under the "Releases" section, and <xref:release-1.0.0> in any markdown page resolves to /releases/1.0.0/.

Verify

  • Run dotnet run --project examples/ExtensibilityLabExample and visit /releases/ — the index lists every entry and each /releases/{version}/ renders.
  • The "Releases" section shows up in navigation with one child per discovered record.
  • Authoring <xref:release-1.0.0> inside a markdown page resolves to the right URL, and the static build (dotnet run -- build) writes one HTML file per route under output/releases/.