The Swift Package Index Blog

Revealing and explaining package scores

Posted Tuesday, October 31, 2023.

This guest post is from Cyndi Chin and announces the results of her work for this year’s Swift Mentorship Program.

Working with Cyndi over the past twelve weeks has been an absolute pleasure. Her enhancements meaningfully push the Swift Package Index forward by making our internal package ranking algorithm transparent, allowing package authors to understand our search results better.

One of Cyndi’s goals was to be involved in shipping something from beginning to end, and the end of that process is the announcement of the feature going live on the site! So, I’ll finish by saying a huge thank you to Cyndi for all of her hard work and hand you over to her for the rest of this post so she can make today’s announcement.

When you search the Swift Package Index, we determine the order in which we show search results by two primary factors: the relevance of the text in the package name and description and an internal score based on various metrics.

Until now, this score was not publicly visible on the site, but as of today, we publish it publicly in a new Package Score section on every Package Maintainer’s page. We also deployed two additional metrics that contribute to the score.

What is a package score?

In combination with the relevancy of a search query, we use a package score to partially influence the ordering of search results on the Swift Package Index. The new information gives package authors and maintainers insight into how we calculate package scores, and the metrics that calculation relies on, such as how actively maintained a package is, whether it has documentation, tests, and various other factors.

The package score section on the package maintainer's page showing a breakdown of package score for a package.

While the algorithm for calculating package scores has always been publicly available, it’s unreasonable to expect people to browse the source code to see how we score packages. Today’s release makes the scoring process much more transparent.

Score traits

The package score is currently evaluated based on ten score traits. Previously, we relied on eight attributes to determine a package’s score.

Adding new score traits

Along with introducing the package score section to the maintainer’s page, we have included two additional score traits in our assessment.

There is always room for further improvement to the package score, and these ten traits are just the start of making this score more comprehensive.

Finding your package’s score

As a package author, if you’d like to see the breakdown of your package score, find the “Do you maintain this package?” section at the bottom of the right-hand sidebar on your package page, and you’ll find the score breakdown at the end.

The location of the Learn More link that takes package authors to the package maintainer’s page for their package.

If you have any feedback regarding the package score or suggestions on how we can improve it, we would love to hear your thoughts in our discussion forum. We appreciate any contributions as we continue to enhance the package score system.

As a final note to this post, I can't express how thankful I am to have had Dave as my mentor in the Swift Mentorship program. Contributing to this project under his guidance has been an incredible learning experience and I'm grateful for the knowledge, support, and inspiration he has provided me. Thank you Dave!

Supporting visionOS

Posted Thursday, June 29, 2023.

Apple was quick to release the promised visionOS SDK only a week after WWDC 2023, and today we’re happy to announce that we have already started compatibility testing for this exciting new platform.

That means we’ve begun building all packages in the Swift Package Index with the visionOS SDK released earlier this week.

You can find visionOS compatibility information in both the build matrix on the package page and our shields.io build badges.

A build compatibility matrix showing compatibility with visionOS.

Note: Just with the Swift 5.9 builds, it will take a short while for our build system to process the backlog of visionOS builds for all packages. Since we are only building visionOS for Swift 5.9, this backlog is much shorter, and we expect to fill in the question marks in the visionOS column of the build matrix swiftly over the coming days.

Supporting Swift 5.9 Betas

Posted Monday, June 12, 2023.

Back in April, we were proud to announce that we supported Swift 5.8 within one week of the official release, adding it to our compatibility matrix so you could know whether the package you were considering would work with the latest version of Swift.

How could we improve on adding support for a new Swift version after just one week of it being available? We could add support for Swift 5.9 one week after the release of the first beta version!

So, that’s the announcement! The Swift Package Index now supports Swift 5.9 packages and will soon show macro products contained within them.

A build compatibility matrix showing compatibility with Swift 5.9.

We also support building documentation with Swift 5.9, but as it’s still in beta, we have made that opt-in for now. If you’d like to build your package documentation with 5.9, you can let us know via a .spi.yml manifest file in your package’s repository.

Note: For the next few days, you may see question mark icons against the Swift 5.9 beta column in the matrix. Please bear with us while our systems process the build backlog.

Validating Swift Package Index Manifest Files

Posted Monday, April 24, 2023.

The Swift Package Index file .spi.yml file allows package authors to configure some aspects of how we process your package. This file has become much more popular since we launched hosted documentation.

Everyone’s favourite config file format YAML of course, comes without any sharp edges and so it is entirely superfluous that we’ve added an online validation tool where you can verify your manifests:

SPI Manifest validation

We hope this helps package authors successfully set up their Swift Package Index manifests!

The Swift Package Index manifest validation page.

Supporting Swift 5.8

Posted Monday, April 3, 2023.

Last Thursday, Apple released Swift 5.8, and today we are delighted to announce that the Swift Package Index already has full support for parsing and processing Swift packages using Xcode 14.3 and Swift 5.8 across all our supported platforms.

Please feel free to add any packages that use Swift 5.8-specific features!

Note: For the next few days, you may see question mark icons against Swift 5.8 in the compatibility matrix. Please bear with us while our systems process the almost 67,000 builds needed to determine package compatibility with a new version of Swift!

A build compatibility matrix showing an unknown compatibility state against Swift 5.8.

Welcoming Apple as a supporter of the Swift Package Index

Posted Tuesday, March 14, 2023.
The Swift Package Index logo next to the Apple logo

As we approach our third anniversary working on the Swift Package Index, it’s great to see the project grow. We’ve continued adding features and working with the community to showcase the incredible work that open-source package authors contribute to the Swift ecosystem.

It has been a long time since we were able to think of this as a “side project“, and as the site becomes a more significant part of this community, the maintenance commitment that the project requires also grows.

Until now, we have run the project with the generous support of our GitHub Sponsors and the support of commercial partners, Stream, Emerge Tools, MacStadium, and Microsoft Azure, covering the site infrastructure and a portion of our time keeping the site operational and moving forward. However, this support only covers a fraction of the time we spend working on the project.

Today, we’re thrilled to welcome another sponsor: Apple. They have generously added their public and financial support to the project, and we couldn’t be happier.

Apple’s support and the community support we already enjoy via your GitHub sponsorships have set us on a path where the Swift Package Index can be a project that fully supports our work financially. It’s also wonderful to have Apple’s support of this community-built, open source project.

We can’t wait to work with Apple and the rest of the Swift community to build the very best place to find Swift packages.

You can see all of our supporters on the Supporters page and there is an announcement post on the Swift.org blog, too.

Recognising Package Authors

Posted Wednesday, December 14, 2022.

When we launched this site back in June 2020, one of the features on the “must be done before launch” list was support for showing who wrote each package. Not just a GitHub username but the names of the primary contributors.

We didn’t quite get that feature in for launch, and then other things took priority as we started dealing with real users and the day-to-day running of the site. Then we tackled build compatibility, documentation, and many other features.

We never returned to author information. Until now!

The Swift Package Index project participated in the Swift Mentorship Program again this year, and we’re delighted to say that Javier Cuesta has done outstanding work resurrecting this feature from the depths of the issue backlog!

The Swift Package Index site showing author metadata for the fastlane package.

By default, we try to determine who has contributed the most to a package and include their name(s). Then, if there are more contributors, we’ll say how many.

All this information comes from git history, and the automated mechanism and algorithm we have used may not get it right. With that in mind, we also allow package authors to override the display of author information through the .spi.yml file.

Adding a top-level metadata key with an authors key below it will override any automatic author information entirely. For example:

version: 1
metadata:
  authors: “Written by Person One, Person Two, and other contributors.”

We want to thank everyone who contributes to open-source software, and we hope this feature helps authors get credit for their efforts.

We’d also like to thank Javier for his hard work and always-positive attitude on this feature. Every aspect of working with him was a pleasure, and this feature only exists today because of his efforts.

Swift Package Indexing - Episode 13

Posted Thursday, October 20, 2022.

In this week’s news, we talked about the extension of our new "universal documentation" links now supporting linking to specific articles or symbols, the new documentation of the .spi.yml manifest file, and how Swift Argument Parser makes creating command line tools a breeze.

We also mentioned Raycast and its Swift Package Index extension by Maxim Krouk. Search got even better this week, too, as we deployed some improvements.

Our quiz segment made a return to cover Swift 5.7 statistics, like how many 5.7-only packages there are and what 5.7 compatibility looks like across the ecosystem.

Our package recommendations this week were:

As always, if you missed this episode you can listen to the recording, and don’t forget to set a reminder to join us for our next episode on the 3rd November.

We’ll speak to you then!

Externally hosted package documentation

Posted Monday, October 17, 2022.

UPDATE: The information in this blog post is superceded by our official documentation. Please refer to the documentation rather than this blog post.

We have a little more documentation-related news to announce today!

We love that so many package authors have chosen to host their package’s documentation with the Swift Package Index. There are now more than 180 packages that have opted in for us to host versioned documentation, and we think that’s excellent news for the Swift ecosystem.

Our hosted documentation generation is the easiest way to get DocC documentation hosted on the web, but some projects have more complex requirements or well-established documentation websites. So, we’re pleased to announce that the Swift Package Index now supports external documentation links!

If you’re a package author and would like to configure your package page to link to your external documentation, add or modify the .spi.yml file in your package’s repository root and tell us where to redirect visitors looking for documentation.

Here’s how to do it:

version: 1
external_links:
  documentation: "https://example.com/package/documentation/"

Here’s an example from GRDB, who already opted into this new feature.

In terms of what visitors to package pages see, it makes no difference where the documentation Lives. Visitors still see a single “Documentation” link in the package page sidebar.

We also created a universal documentation link for any package that has documentation either hosted by us or externally. For any package with documentation, navigate to:

https://swiftpackageindex.com/{owner}/{repository}/documentation

Replacing {owner} and {repository}, of course, and it will redirect to either our hosted or the external documentation.

Swift Package Indexing - Episode 12

Posted Thursday, October 6, 2022.

We started this episode, as usual, with some news from the project. We talked about generic documentation links (so you can share a link to your package’s documentation without encoding a branch or tag version in the URL), the completion of our Swift 5.7 build processing and the new DocC sidebar. Then we chatted about the Thread Sanitizer (TSAN) and Swift on the Server projects and more virtualization talk!

We also spoke about the Package Registry Service proposal and its reference implementation, how a registry differs from an index, what package identifiers will bring, and more.

Our community packages for this week were:

If you missed it, the recording is available here. We hope you’ll join us for our next episode on the 20th October, too!

Swift Package Indexing - Episode 11

Posted Thursday, September 15, 2022.

We just wrapped up our latest Twitter Space. This is becoming a habit!

We started with our usual discussion of what we’ve been up to with the Package Index, focusing primarily on the compatibility build system, and the Swift 5.7 build backlog. We also discussed external documentation hosting for package authors, and our internal package scoring system.

If you are curious about the performance of virtual machines created with the Virtualization framework you’ll hear us mentioning some surprising results, discovered while using VirtualBuddy by Guilherme Rambo.

Then, our package recommendations:

You can listen to the Space recording if you missed it live, or join us for our next scheduled space. We hope to see you there!

Swift Package Indexing - Episode 10

Posted Thursday, September 1, 2022.

Another episode of Swift Package Indexing just finished, and we managed to keep the chatter about next week’s announcements from Apple limited to a couple of terrible jokes!

We stuck to our usual format, starting with some site news, then did some quiz questions, and wrapped things up with package recommendations!

Quiz

This week’s quiz questions were about analytics! Specifically where our visitors are from. We talked about countries, regions, and cities.

If there’s any question you’d like us to answer on a future episode, tweet at us at @SwiftPackages.

Packages

Here are the packages we talked about this week:

I think it’s also about time we had another guest on the show. Would you like to come on and talk about a package you created or maintain that has had a recent release? Tweet us at @SwiftPackages and let us know you’d be interested!

The recording of this episode is available here, and we’ve scheduled the next episode for the 15th of September at the usual time. Set yourself a reminder!

Swift Package Indexing - Episode 9

Posted Thursday, August 18, 2022.

We just wrapped up Episode 9 of Swift Package Indexing, which was another chat about what’s going on in the world of Swift packages!

As usual, we started with some updates about the Package Index itself. Bugs and improvements to DocC documentation, a relaxation of the rules for what types of packages belong in the index, improvements to search by Joe Heck (and the fascinating discussion in #1876), and some upcoming changes from Javier Cuesta around acknowledging package maintainers and authors!

We then had our usual Quiz and Package recommendations:

Quiz

Here are this episodes questions, asked via Twitter:

If there’s any question you’d like us to answer, please tweet at us at @SwiftPackages.

Packages

If you missed the recording, catch up here, or join us for our next episode on the 1st September!

Versioned DocC Documentation

Posted Friday, August 5, 2022.

UPDATE: The information in this blog post is superceded by our official documentation. Please refer to the documentation rather than this blog post.

We rolled out auto-generating, auto-hosting, and auto-updating DocC documentation exactly two months ago, and here we are again with more documentation-focused news!

Opting into this feature was simple for package authors with a quick amendment to the package’s .spi.yml file telling us what targets/platforms to use for the documentation generation process. We handled everything else from there, and a “Documentation” link automatically appears for every opted-in package.

We’re delighted to say that this feature has been a great success, and we now have over a hundred packages that have chosen to host documentation with us. That’s amazing!

We have some good news for those 103 package authors today. You made the right choice because you now automatically get versioned documentation, too!

We initially generated documentation for the default branch of a package. It was the easiest way for us to build the feature and a good first step, but versioned documentation is what we were aiming for.

When you use a package, you’d typically use the latest tagged release, so we should show documentation generated from that release by default. As of today, that’s what happens:

Package documentation showing a tagged release in the breadcrumb bar at the top of the page.

We continue to generate documentation for the default branch, and we’ll keep archived documentation available for every major version as packages evolve. Hover over the release to switch to the documentation for a different release or for the default branch:

Package documentation showing a drop-down menu with both the latest tagged release and the default branch version.

We also moved the selector for packages that generate multiple documentation archives for one package:

Package documentation showing a drop-down menu with multiple documentation archives generated from one package.

Why not check out the feature with one of the documented packages? In alphabetical order, you can pick from AdvancedList, AgoraUIKit_iOS, AnimationPlanner, AnyAsyncSequence, AnyLint, AnyMeasure, apnswift, AppFoundation, AppStorage, Boutique, Buildkite, bytes, CalendarKit, CameraControlARView, CertificateSigningRequest, CombineCoreBluetooth, composable-effect-identifier, Compute, ConcurrencyPlus, CryptoSwift, CurrencyConverter, CustomRepeatDate, DesignReviewer, Dflat, EffectsLibrary, Epoxy, FileSystemEventPublisher, FocusEntity, GatheredKit, GeoJSONKit, gis-tools, HandySwift, HandyUIKit, KeyboardShortcuts, KeyboardToolbar, Kingfisher, LeftPad, Lindenmayer, LocationFormatter, Lottie, Media, MeshGenerator, Microya, MonthYearWheelPicker, mqtt-nio, MuJoCo, mvt-tools, NetworkReachability, OperationPlus, ParseSwift, PianoRoll, Pioneer, postgres-nio, Pretty, PrettyLog, reactiveswift-composable-architecture, RevenueCat, Runestone, Saga, ScaledFont, SceneKitDebugTools, Script.swift, secp256k1, SemanticVersion, SemanticVersioningKit, Server, SGPKit, SmoothGradient, SpanGrid, Squirrel3, Steam, StoreFlowable, StreamChat, StreamChatSwiftUI, StreamChatVapor, Stytch, swift-bundler, swift-case-paths, swift-composable-architecture, swift-confidential, swift-markdown, swift-multipart-formdata, swift-parsing, swift-url-routing, swift-xml-parser, SwiftDocC, SwiftInspector, SwiftPlantUML, SwiftProtobuf, swiftui-cached-async-image, SwiftUICharts, SwiftUnits, SwiftVizScale, SwiftyProvisioningProfile, TGCardViewController, UnsplashFramework, Updeto, URLCompatibilityKit, vapor-routing, WeakReference, XCSnippets, Yams, or YMatterType! Phew! 😅

We hope everyone loves this feature, and if you want to generate docs for your package, instructions are in the launch blog post.

Swift Package Indexing - Episode 8

Posted Thursday, August 4, 2022.

We just wrapped up another episode of Swift Package Indexing with the usual mix of news, Swift Package Index progress, and other interesting bits from the last couple of weeks.

We talked about:

We also, as always, highlighted some packages:

You can listen to the full recording here and you can set yourself a reminder and join us live on the 18th of August!

Swift Package Indexing - Episode 7

Posted Thursday, July 21, 2022.

We just wrapped up another episode of Swift Package Indexing! We talked again about the progress we’ve made working towards versioned documentation hosting. Sven had more quiz questions for Dave, and of course, we talked about community packages!

The packages from this week were…

The Twitter Spaces UI continues to baffle Dave, and yet again he messed up the recording. 🙄 There’s very little he can be trusted with these days. Apologies for that! It’s definitely set for next time, though. (I’m absolutely certain I set it for this time, too 😬).

Set yourself a reminder, and join us live on the 4th August!

Searching for plugins?

Posted Tuesday, July 12, 2022.

With Swift 5.6, Xcode and the Swift Package Manager gained a new product type, plugins, allowing developers to extend their build process with new build commands or processing steps.

Here are two WWDC 2022 sessions that go into more detail on what plugins are and how you can create them:

What use are plugins? Rather than link to some examples directly, let’s use the latest feature of the Swift Package Index to find out! You can now filter search results based on whether packages include a plugin.

Search for packages containing a plugin

As you’ll notice, there are packages containing plugins to help you generate documentation (and you can find out more about that here, too), integrating SwiftLint, and a few more!

This feature has been a group effort by several contributors! Joe Heck first suggested it would be a helpful feature, and Marin Todorov took up the challenge and implemented it! We’re so happy to accept this contribution which means we have support for plugins just a few weeks after Apple officially introduced them at WWDC.

Thanks to open source contributions, our search is improving!

We need your help to fund this project

Posted Monday, July 11, 2022.

As an open-source project without the backing of a big company, community donations are critical to keeping this project going.

Dave wrote more about this in this week’s iOS Dev Weekly if you want the whole story, but the short version is that this project needs your help.

Every contribution helps, whether it’s a one-time or monthly donation. Every sponsorship directly supports the time Dave and Sven dedicate to this project.

We’re in this for the long term to build the package index that the Swift package ecosystem needs, but we need your help. Please consider joining the 99 other people who sponsor this project through GitHub sponsors.

A grid of profile pictures showing the current sponsors of the Swift Package Index.

Thank you so much!

Swift Package Indexing - Episode 6

Posted Thursday, July 7, 2022.

No guest this week, so the show was back to Dave and Sven talking about all things Swift packages! We discussed some of the open-source contributions we’ve received over the last few weeks and some more upcoming work on hosted DocC documentation.

Sven also surprised Dave with a Swift Package Index trivia quiz! Do you know what percentage of packages have zero open issues or who has the most packages in the index? There’s only one way to find out!

Of course, we also had some package recommendations and details about some significant releases from the last two weeks:

No recording mishaps this week, so you can listen to the full recording here, and if you want to join us next time, we’ve already scheduled the next episode for two weeks time. Set a reminder and join us!

Swift Package Indexing - Episode 5

Posted Thursday, June 23, 2022.

This week we talked with Corey E. Baker about ParseSwift, developing open-source, and what the next generation of computer science students are interested in. We also had some package recommendations, as always!

Here are the links to what we talked about:

Unfortunately, there was a mishap with the recording of this episode (and by that, I mean we didn't record it 😅), but if you want to join us next time, we’ve already scheduled the next episode for two weeks time. Set a reminder and join us!

We'll double-check the recording settings next time, too! 🙈

Swift Package Indexing - Episode 4

Posted Thursday, June 9, 2022.

We’re two months into our little experiment of bi-weekly Twitter spaces discussing everything Swift Package Manager and Swift Package Index, and we’re really enjoying the format. Do you have any feedback on how we’re doing with it? We’d love to hear from you with feedback, suggestions for topics, guest recommendations, or just packages you love that we should feature!

This week’s episode was all about some of the new technologies at WWDC, Swift 5.7, and some package recommendations. Here are the links to what we talked about:

You can listen to the full recording here, and if you want to join us next time, we’ve already scheduled the next episode for two weeks time. Set a reminder and join us!

Auto-generating, Auto-hosting, and Auto-updating DocC Documentation

Posted Friday, June 3, 2022.

UPDATE: The information in this blog post is superceded by our official documentation. Please refer to the documentation rather than this blog post.

Introduced at WWDC 2021, DocC is Apple’s recommended way to provide documentation for your packages.

It’s easy to use, and the resulting documentation looks great. It generates documentation either from comments or separate article files written in Markdown that is more suitable for longer-form documentation. You can even use it to create beautiful interactive tutorials with images and step-by-step instructions. DocC generates either an Xcode documentation archive or a set of HTML and CSS files that you can host on a web server.

Of course, having a directory full of HTML is only half the battle. Your next task is to get it hosted somewhere online and maybe even set up a CI task to automate that process so that your published documentation stays up-to-date as your development progresses.

That’s where our latest feature will come in handy, and we’re launching it today!

Auto-generated, auto-hosted, and auto-updating

Our build system can now generate and host DocC documentation and make it available from your package’s page in the index. All we need is a little configuration data so that we know how best to build your docs.

Once configured, you will see a new “Documentation” link in the sidebar and never have to worry about your documentation again!

The DocC package page showing a link to the auto-generated and hosted documentation.

As a package author or maintainer, there are only threetwo things you need to do for the Swift Package Index to build and host your documentation.

  1. Ensure that your package builds successfully with Swift 5.6. Your package can support earlier versions of Swift, too, but must successfully build with 5.6.
  2. Add the swift-docc-plugin documentation plugin to your package’s Package.swift manifest file, if you haven’t done so already.
  3. Create a .spi.yml file and commit it to the root of your package’s repository, telling our build system which targets have documentation.

From there, we’ll take care of everything else. Every time you push new commits to your package, we’ll regenerate your documentation.

Adding the Swift DocC plugin

UPDATE: It’s no longer necessary to add the DocC plugin to your package. If you’ve added it already, feel free to remove it. Our build system will also cope if you leave the plugin or need it for another purpose, so there’s no rush to remove it! The only change you need to make to your project is to add the manifest file.

You may already have taken this first step if you’ve worked with DocC locally. If not, then add the following lines to the end of your Package.swift:

#if swift(>=5.6)
  // Add the documentation compiler plugin if possible
  package.dependencies.append(
    .package(url: "https://github.com/apple/swift-docc-plugin", from: "1.0.0")
  )
#endif

Making the plugin dependency conditional means that nothing will break even if the swift-tools-version of your manifest is for an earlier version of Swift than 5.6. Naturally, this is unnecessary if your package only supports Swift 5.6 and above.

Add a Swift Package Index manifest file

Then, create and commit a manifest file named .spi.yml file in the root of your repository that looks something like this:

version: 1
builder:
  configs:
    - documentation_targets: [Target]

This file tells our build system to generate documentation for the target Target. You would typically set this as your package’s main (or only) target, but it may also be a dedicated target containing documentation Markdown files.

You can also specify multiple targets, and we’ll add a target switcher in the hosted documentation so people can easily find all your documentation!

Documentation platform

By default, we will generate documentation using macOS. If your package requires the documentation generation to be run for a certain platform such as iOS, you can also specify a platform:

version: 1
builder:
  configs:
    - platform: ios
      documentation_targets: [Target]

Auto-updating frequency

To keep the amount of processing that our build servers perform under control, we only build the default branch for each package at most once every 24 hours. So, when you push the configuration file live, the system will generate that set of documentation, but it will then be 24 hours until the generation process runs again. If there have been any commits during that period, we’ll create docs from the latest commit when the period resets.

Hosted documentation for the DocC package shown in the context of the Swift Package Index with a header above the documentation.

Initial adopters!

Note: We'd like to thank the following package authors for coming on board with our documentation hosting feature so early. The feature is fully released and stable now and available for all package authors. for more information, see our SPIManifest documentation.

You may have seen a call for package authors with DocC compatible documentation earlier this week, and we’re thrilled to say that we have 20 packages that have already added configuration files and have their documentation hosted by us! Why not check them out?

If you maintain one of the 4,600+ packages we have in the index, please add your configuration file to opt-in to having your documentation hosted on the Swift Package Index, and we’ll take care of everything else. If you have any issues, please join us on Discord or raise an issue.

Next steps

We’re proud of what we have built here, but we’re not done with this feature yet.

There’s a write up of things we want to tackle next on our Discussion forum, but to summarise, this is what we’re planning to work on next:

Swift Package Indexing - Episode 3

Posted Thursday, May 26, 2022.

We just finished our third Twitter space, where we had Simon Støvring join us to talk about his recently released package Runestone, as well as some other package releases that caught our eye since the last episode.

You can find links to Simon's work on his website, his Github profile, and follow him on Twitter.

This week, we also talked about all of these wonderful packages:

You can listen to the full recording here, and if you want to join us next time, we’ve already scheduled the next episode for two weeks time. Set a reminder and join us!

Swift Package Indexing - Episode 2

Posted Thursday, May 12, 2022.

We just finished our second Twitter space, where we talked about some recent package releases that caught our eye since the inaugural episode two weeks ago.

This week, we talked about all of these wonderful packages:

You can listen to the full recording here, and if you want to join us next time, we’ve already scheduled the next episode for two weeks time. Set a reminder and join us!

Swift Package Indexing - Episode 1

Posted Friday, April 29, 2022.

We tried something new for the Swift Package Index this week, a public discussion using Twitter Spaces where we talked about some of the package releases from the community.

It was an experiment that we deliberately didn’t announce or advertise, so don’t worry if you missed it. We were curious whether a discussion format would work for talking about package releases and didn’t want to make a big deal out of it in case it didn’t! Even without promotion, we still ended up with 115 people joining us at some point during the 45-minute discussion, though!

The good news is, it did work, and we’ll be doing it again. We’ll tweet about it a couple of times before it happens, so the best way to keep up is to follow us on Twitter.

You can also listen to the recording!

We also wanted to link to the packages we highlighted during the discussion, so here goes! We talked about:

Thanks to all the authors and maintainers of these packages for their hard work!

Two years of the Swift Package Index

Posted Monday, April 25, 2022.

It’s been exactly two years since the first commit¹ of the Swift Package Index! More than 950 pull requests and 5,250 commits later, we now index and compatibility test over 4,500 packages!

The site’s goal has always been to help you make good decisions about the dependencies you include in your projects, and that goal remains today. Including third party code in your app is a delicate balance between future maintenance of that dependency and the time it can save you today. To make those decisions, you need lots of metadata, and that’s what we’re trying to provide.

A milestone like this is also an excellent time to mention that this project is open-source and open to contributions! We’ve had many valuable contributions in the past couple of years, but there’s plenty more to do. We’ve marked some issues with the “good first issue” tag if you’re looking for a place to start. We’d also love to welcome you over on our Discord if you want to contribute.

So, I hope you’ll join me in wishing the Swift Package Index a happy birthday! Helping to create something for the community has been a great experience so far, and we’re excited about where we can take this project in the future! 🥂

¹ In March 2020, we experimented with a Vapor 3 version of the project, and there was the SwiftPM Library before that. The first commit of the software that became what you use today was made on the 25th of April 2020, though.

We support Ukraine

Posted Saturday, February 26, 2022.

Russia’s invasion of Ukraine is appalling.

We wanted to make a contribution to support the humanitarian efforts of the Red Cross, so we have donated and personally matched all of this month’s sponsorship from the Swift Package Index to the Ukraine Emergency Appeal from the Ukrainian Red Cross Society.

We wish a safe and swift resolution to this situation to all our friends in Ukraine. 🇺🇦

Keeping up with dependency changes

Posted Tuesday, January 25, 2022.

How often do you update your package dependencies, and when you do, do you check what’s new or changed, or do you just run your tests to check nothing broke and move on? 😅

Don’t be embarrassed. You’re not alone! Even if the package authors have lovingly crafted a detailed change log, checking them for every package you use is painful. It’s no wonder no one takes the time to do it.

So we built a little tool to make the task easier. Install the swift-release-notes tool, and get a list of links to the release notes for all the packages that have pending updates.

$ swift release-notes ~/Projects/SPI/spi-server

(... progress output removed)

Release notes URLs (updating from):
https://github.com/vapor/fluent-kit/releases (1.19.0)
https://github.com/apple/swift-llbuild/releases (main)
https://github.com/vapor/vapor/releases (4.54.0)
https://github.com/apple/swift-package-manager/releases (main)
https://github.com/vapor/async-kit/releases (1.11.0)
https://github.com/apple/swift-nio-ssl/releases (2.17.1)
https://github.com/apple/swift-tools-support-core/releases (main)
https://github.com/apple/swift-nio-transport-services/releases (1.11.3)
https://github.com/apple/swift-driver/releases (main)
https://github.com/apple/swift-nio/releases (2.36.0)

Give it a try and be better aware of what new changes are happening in the packages you’re using!

Improvements to package search

Posted Wednesday, December 15, 2021.

Recently, we’ve been focusing our attention on improving search here on the Swift Package Index, and it’s time to let you all know what we’ve implemented!

Search Results

First up, we improved the information you’ll see when you see search results or any list of packages:

Search results that include the number of stars a package has and when the repository last had maintenance activity, in addition to the standard metadata.

All package lists across the whole site now include the number of stars and how recently a package has had maintenance activity so you can start making some decisions before you even open a package page. You can see examples of this in search results, keyword results, and owner/author pages.

Note: We update the date of the last “maintenance activity” when there is a commit to the default branch or when someone closes or merges an issue or pull request.

Search Filters

We’ve also added filters so you can refine any set of search results. The best way to explain this feature is with some examples:

Head over to the documentation for more information on all the fields you can filter on and more information on this significant feature.

Search results that have filters applied to narrow down the results based on last maintenance activity and platform compatibility.

We’re not entirely done with updates to search yet, but what we have is certainly significant enough for an update post!

As always, all of this work is a team effort, and these improvements would not exist without the invaluable contributions of James Sherlock, who did the lion’s share of the work getting filters implemented and Sarah Lichter, who helped with the package list improvements. Thank you both!

M1 Pro and M1 Max Xcode Build and Test Benchmarks

Posted Monday, November 1, 2021.

It is a truth universally acknowledged that a developer in possession of a good project must be in want of a fast compile. First observed some 200 years ago by Jane Austen, this still holds today as lucky developers take delivery of new M1 machines.

You’ll have seen benchmarks appearing on various sites over the last week, but how do these machines stack up with a real-world Xcode project?

The Swift Package Index is a sizeable open source project and makes for a good test when assessing the new chips, so we’ve created a benchmark tool to give some real-world results.

The Tests

We are testing two performance aspects: building the project (clean build) and running the tests. These tasks might seem similar, but they stress the machine in quite different ways. A clean build will typically utilise all available cores, where this project’s tests use a dockerised database running on the same machine and are not heavily multi-threaded.

First, we know what you all want to see. The results!

A table of build benchmarks showing a dramatic reduction of build times from Intel MacBook Pro machines through to M1-based machinesA chart showing test benchmark results increasing steadily from the fastest M1 Max machine to the slowest Intel-based machineA chart showing test benchmark results increasing from one set of times for the M1-based machines to another set of resuklts for Intel-based machines

Overall, it’s remarkable that the M1 MacBook Air already had the best performance before Apple introduced the new MacBook Pros, but the M1 Pro and Max chips take this further. They improve on the M1 Air’s best result of 47 seconds with a build time of less than 31 seconds. Those extra cores matter, and the ~35% improvement is in line with what you’d expect, going from a 4+4 performance/efficiency core setup to an 8+2 configuration.

If you’re considering upgrading from an Intel machine to an M1 Pro or Max, you’ll see a dramatic reduction in build times. It’s been a long time since we’ve seen 2x improvements in a machine of the same class, but the best Intel build time is 60 seconds versus the above mentioned 31 seconds of the M1 Max. That is remarkable.

The results are much less dramatic when running the tests. The M1 Max’s extra cores don’t help much, and the best time between the regular M1 and the Pro/Max is inconsequential. Just less than 27 seconds on the M1 MacBook Air and just less than 26 seconds on the Mac Book Pro.

M1 Pro vs M1 Max, 32Gb vs 64Gb, and 8-core vs 10-core

You may have noticed no difference between the benchmarks when switching between the M1 Pro and the M1 Max. The GPU is not used for compilation or in these tests, so it makes perfect sense, but it’s worth mentioning. It’s also worth noting that we saw no difference between an M1 Max with 32Gb and one with 64Gb, difference of 0.2 seconds on builds and 0.6 seconds on tests is well within the error margins.

There was a noticeable difference between the M1 Pro 8-core and the M1 Pro 10-core machine, with the 10-core being around 15% faster when building. That’s quite significant but is in line with what you might expect from dropping two performance cores. As mentioned above, we have ignored the test benchmark results for this machine as there may have been a problem with the benchmarking environment for this machine.

There are plenty of reasons to get a machine with a Max chip and 64Gb, but compiling your code faster is not one of them!

M1 MacBook Air Thermals

If there’s one thing where the MacBook Air suffers, it’s the sensitivity to a warm environment. The fact that the machine comes without a fan is excellent for its guaranteed silence but leads to a noticeable slow-down when used in warm weather, direct sunshine, or when you stress the CPU over extended periods.

You can even see a little of the effect by observing the slight upwards slope of the build benchmarks for the M1 MacBook Air.

A chart showing just the build benchmarks for the M1-based MacBook Air

During the 15 minutes that this benchmark ran for, the machine got quite warm to the touch, and the chart shows that it’s starting to throttle a bit.

Verdict

If you’re using an Intel machine, even the very fastest ones that Apple ever made, it looks like you can expect your builds to be at least twice as fast. Add to that the larger and better screen, the ports, and other minor improvements, and this feels like the best upgrade we’ve seen for developers who prefer a laptop in at least a decade.

If you’re on an M1 machine, the performance is less striking as the individual cores on these new chips have almost identical performance. However, you’ll still see some improvements due to the increased core count and a higher percentage of performance cores.

All of that said, it’s worth remembering one thing, especially if you’re on a tighter budget. The M1 Max MacBook Pro with 64Gb is 100% faster than the fastest Intel machine we tested (30.9 seconds vs 60 seconds), where the M1 MacBook Air is 30% faster than it. However, the M1 Max MacBook Pros tested here all sell for around ~$4,000, and the M1 MacBook Air tested here was ~$1,600. Performance does not scale linearly with price.

There was a time where if you recommended a low-end MacBook to a developer who was just getting started, you’d know that it would be adequate but never great. That’s no longer true, and even the lowest-end M1 machine will have comparable performance to the highest-end laptops.

What should you buy?

If you have a budget of less than $2,500, buy an M1 MacBook Air (not the 13" Pro. The MacBook Air is the bargain here!). If you have more than $2,500, then buy the 10-core M1 Pro.

Thanks

Thanks to Christian Weinberger, Toby Herbert, Greg Spiers, and Cameron Cooke for taking the time to run the benchmarks that covered the configurations that we didn’t have available!

Also, if you'd like to look more closely at the results, all of the raw numbers referenced in this post are available for download.

Package Dependencies - Step One

Posted Monday, October 18, 2021.

We’ve said several times on this blog that one of the primary goals of the Swift Package Index is to help you make better decisions about the dependencies you include in your Swift projects.

In pursuit of that goal, we just launched a new feature that shows you how many dependencies each of the packages in our index has. 🚀

Even better, it doesn’t just give you the number of top-level dependencies that a package declares in its Package.swift file. Instead, we inspect the Package.resolved file after the build system has done the hard work of dependency resolution.

Package metadata showing that the package being inspected depends on nine other packages.

That said, while this feature adds value and we’re happy to get it launched, this is step one towards where we’d like to be with dependency metadata. Some things to note:

However, what we have for now is still a step in the right direction!

Launching Swift Package Collections

Posted Wednesday, June 9, 2021.

It's probably not news that this is a big week for Swift developers! On Monday, Apple kicked WWDC off by releasing the first beta of Xcode 13, including Swift 5.5. We've been following the release of the Swift Package Manager in Swift 5.5 closely as it implements package collections, a significant new feature.

What are package collections? JSON descriptions of Swift package metadata. What does the Swift Package Index have in droves? Metadata about Swift packages! As soon as we heard about the proposal, we knew we had to support it. For a comprehensive description of the feature, check out the post over on the official Swift blog.

We have big plans for package collections, but we wanted to ship something alongside this first beta release. We chose to implement author collections, a list of all packages from the index grouped by repository owner. For example, you can get a package collection containing all packages from Alamofire or all packages from Point-Free.

We expected this feature to roll out as part of the Swift Package Manager command-line tool, and it did. What we didn't expect was full support for package collections in Xcode 13! What a pleasant surprise that was. 😍

We've written up the details of how to access collections on the Swift Package Index, including how to add them to Xcode and use them from the command line. We'd love for you to check it out.

However, that wasn't the only surprise this week had in store for us! As we watched Nicole Jacque talk about What's new in Swift, we were amazed to see a screenshot of the Swift Package Index appear on the screen and hear Nicole start to talk about it. Remarkable!

The Swift Package Index site being presented in What's new in Swift at WWDC 2021

We're so happy that Apple sees what we've been working on here as important enough to mention in a WWDC session, and we'd like to thank everyone who was involved in that decision for including us. It's a great honour and one that we do not take lightly.

What’s Next? - May 2021

Posted Tuesday, May 18, 2021.

You might just have finished reading our progress report from the last three months. Now it’s time to talk about what we’re planning for the next few months!

Like last time, this is a list of areas that we’ll focus on over the next few months, not a to-do list. Other features and bug fixes will crop up and might take priority, and that’s OK! It’s good to have a rough direction in mind, though.

Author Information

We feel strongly that open-source package authors should be acknowledged for their work. The very first designs for the package index had author information right at the top of the package metadata, but it didn’t quite make it into our feature set at launch. We want to change that, adding author information prominently at the top of our package metadata section.

We’ll collect the author information via the .spi.yml file we already use for build configuration.

Package Keywords

We’ve long debated whether packages should have categories that come from a pre-determined list or have freeform keywords. After giving it some thought we’re going to go with freeform keywords rather than categories. We’ll collect keyword metadata from GitHub topics and show the keywords on package pages.

Dependency Information

We’d also like to start working towards displaying the dependencies that a package would bring into your project if you use it. For example, we might start by adding a simple count of top-level and total dependencies.

Additional Package Metadata

We’d also love to pick up a few more data points about every package. Does a package have tests? Is source code available? Is it a fork of another package? All of these are good indicators when considering dependencies.

Home Page Redesign

Using the author information and keyword information mentioned above, we’d love to work on a much more information-rich home page. The first part of this is to think about it properly and put together a design, which is why we’re not going to talk about precisely what you can expect today. However, we’ll be working on this.

Package Collections

Finally, we’d like to finish off our support for Package Collections and put it live to coincide with the release of Swift 5.5. Initially, we’ll support collections that show all packages authored by their repository owner. So, for example, a collection of these packages from Apple, or these packages from the Vapor project.

Progress Report - May 2021

Posted Tuesday, May 18, 2021.

Around three months ago, we published a list of areas that we wanted to focus on next. We’re just about to update that list for the next few months, but before we do that, let’s check in with how well we stayed on track since February!

We’re pretty happy with that progress, and that list doesn’t cover all of the other things we did, such as adding build support for Swift 5.4. 🚀

Of course, none of this would be possible without the generous support of our GitHub Sponsors. We’d first like to thank all of our new sponsors since February. Adam Fowler, Gaston Glyn Rampersad, Joakim Hassila, Joohee Kim, Loopwerk, Marius, Omar Albeik, Point-Free, Raymund Beyer, and Thomas Prezioso Jr.

Of course, we also want to show our continued thanks to our ongoing sponsors! Alexandre Colucci, Artem Loenko, Bartosz Dolewski, Boris Bügling, Byron Davies, Chris, Dominik Hauser, Federico Zanetello, Gleb Shevchenko, James Sherlock, Jason Bruder, Jeffrey Macko, Joseph Heck, Joshua Kaunert, Juanpe Catalán, Jules Coynel, Kyle Lee, Leo Picado, Marcos Griselli, Massimo Biolcati, Matěj Kašpar Jirásek, Max Desiatov, Nathan Harris, Oliver Drobnik, PatrikTheDev, Paul Farnell, Paul Hudson, Peter Steinberger, PSPDFKit, Riley Testut, Robin Kunde, Sam Symons, Scott Robbins, Thomas Dohmke, Tim Condon, Umar Haroon, Vadim Shpakovski, and Xavier Lowmiller. Thanks so much for your continued support. ❤️

Would you like to support the development of the Swift Package Index? We're almost half way to our first goal of 100 supporters!

So, what’s next? Read on to find out.

Using the SPI Playgrounds app to file better bug reports

Posted Monday, May 17, 2021.

We recently released the SPI Playgrounds app, which allows you to try out Swift Packages in a Playground in Xcode. You can use it to quickly get up and running with a new package when evaluating dependencies. However, it can also be helpful when working with dependencies you already use in your project. One such use case is creating reproducible bug reports.

It’s often tricky for open-source maintainers to help people when they report issues, especially if they are in a large app, where reproducing a bug might require a significant amount of set-up. It can be especially challenging for Swift on the server projects, where you might have dependencies like databases and other services.

To help open-source maintainers to help you, it can be useful to prepare something they can run in a standalone way. How about in a playground? It doesn’t get much simpler than that!

Recently, looking at one of the issues in the Swift Package Index, it turned out that this was the perfect opportunity to use Arena to create a playground where we could reproduce the issue.

Note: Arena is the underlying tool used in the SPI Playgrounds app. Think of it as the command-line version of the app. We’re using it here as it allows you to create a Playground with more than one dependency embedded, which the SPI Playgrounds app doesn’t yet support.

We started by running Arena as follows:

arena https://github.com/vapor/vapor https://github.com/vapor/fluent https://github.com/vapor/fluent-postgres-driver -o ssl-error-repro

This command produced a playground which imported Vapor, Fluent, and the Postgres driver. We knew the issue happened when accessing an Azure Postgres database via TLS, so we set up a test database with a single record and read the record via Fluent.

The whole snippet to reproduce the bug was about 30 lines:

import Vapor
import Fluent
import FluentPostgresDriver

final class Test: Model, Content {
    static let schema = "test"

    @ID(key: .id)
    var id: UUID?
}

func run() throws {
    let app = Application(.testing)
    defer { app.shutdown() }

    let tlsConfig: TLSConfiguration = .forClient()
    app.databases.use(.postgres(hostname: "pgnio-debug.postgres.database.azure.com",
                                port: 5432,
                                username: "test@pgnio-debug",
                                password: "<ask me for the password>",
                                database: "test",
                                tlsConfiguration: tlsConfig), as: .psql)

    let db = app.db

    let t = try Test.find(UUID("cafecafe-cafe-cafe-cafe-cafecafecafe"), on: db).wait()
    print("t: \(String(describing:  t))")
}

run()

In the end, it took more time to set up a new test database for the issue. What’s more, it was trivial from that point on to iterate and try more things as instructed. Much more so than it would have been had it still been part of the Swift Package Index server project.

In some ways, creating this playground helped get this bug fixed. If we had not taken the time to reproduce the issue in a standalone way, it would have been difficult for the maintainers to be sure it was a problem they could fix.

Thanks again to Fabian Fett, Gwynne Raskind and Johannes Weiss for their help and Fabian for the quick turnaround on the fix, and we hope this post also helps you file better bug reports!

Launching the Swift Package Index Playgrounds app for macOS

Posted Wednesday, May 5, 2021.

From the very beginning, the main goal of this site was to do more than echo Swift package metadata. We want to enable better decisions about which dependencies to use. That’s why we include information about how actively maintained a package is, and it’s why we check compatibility with the different platforms by building against them.

Once you’ve decided that a package might be a good fit for your project, what’s your next step? Do you import it and start coding against it?

There’s a better way.

What if you could click a button and open Xcode with a Swift playground that has the library you’re thinking of using already imported? That’s what we’re launching today with the Swift Package Index Playgrounds app.

Once you have installed the app, click “Try in a Playground” from any package page.

The Swift Package Index site with the Try in a Playground button highlighted

Then click "Create Playground" when the app launches.

The Swift Package Index Playgrounds app running on macOS

Choose where to save it, and a few seconds later, the playground will open in Xcode, ready for experimentation and testing! We’d love to know if this is a helpful feature, so be sure to let us know on Twitter.

There’s also an amusing story behind this feature, which prompted us to write up the story behind “The Button”.

How the Swift Package Index project got started because of a button

Posted Wednesday, May 5, 2021.

It’s been one year since we made the first commit on the Swift Package Index repository, and we think that deserves a little celebration!

🎂

Did you enjoy the cake? 😂 No? Well, how about we also tell you the story of how this project started?

Note: We usually write blog posts here in the first person plural as this project is very much a joint effort, but this is a story with lots of references to both Dave and Sven individually, so I’ll write it in the first person singular. Dave

Before the Swift Package Index, there was the SwiftPM Library. The goal of that site was the same as this one, to provide a comprehensive index of Swift packages that helps you make better decisions about your dependencies.

A little while after the launch, I had a message from some members of the Vapor Discord. They had been chatting about creating something similar as an open-source Swift/Vapor project and wondered if I might be interested in collaborating on it. I’ll admit that I wasn’t hugely excited about rewriting a site that I had just launched, but I was in favour of it becoming open-source. However, the reality of an open-source project focused on indexing Swift packages being a Ruby project would make finding contributors challenging, so I put aside my doubts and decided to go for it. I started learning Vapor and got started. Unfortunately, circumstances worked against the rewrite being a success at that point. I found the Vapor learning curve tough while also maintaining the existing site, and the folks from the Vapor Discord were busy too. The project started to progress, but slowly.

That’s when I got an email from Sven. He had seen the SwiftPM Library and wondered if I might be open to integrating his open-source project, Arena, with it. His idea was to add a button next to each search result that, when clicked, would create a Swift Playground that imported the library, ready for experimentation. It was a great idea, and we ended up on a quick audio call a few days later to discuss it. While we chatted, I mentioned the potential rewrite of the SwiftPM Library into Swift/Vapor, and it turned into a much larger conversation. Sven was interested in the project, experienced working with Vapor, and keen to help. A winning combination! 🥇

Things progressed very quickly after that. I wrote up some details of how the existing project worked, and Sven worked his Vapor magic. Before too long, we had a prototype up and running, and the project started to feel real. There wasn’t time to add “the Arena button” as we needed to get the basics implemented first, but it remained firmly on the to-do list.

Sven and I ended up working really well together. We had different, complementary skill sets, and we progressed quickly towards getting the new project finished. It had been open-source from day one, and we were working in public, but no one had noticed, and we started to gear up for the launch of the Swift Package Index.

There was still no sign of “the button”, though, even many months after the launch. 😬

It wasn’t through any malice or deliberate neglect. It was just that there was always something else that took a slightly higher priority. We worked on language and platform compatibility reporting, Apple silicon builds, gathering additional metadata, funding the project, RSS feeds, inline README files, finding suitable hosting, and many more things that didn’t make it to the blog.

I’m delighted to say that today sees the launch of “the button”! But what does the button do? Read more about that in the official launch post.

That deserves more than a cake! 🥂🍾

I’m happy that we finally got to bring Sven’s original idea to fruition through this project, but this celebration is about much more than that! Working with Sven on this project has been a pleasure from the beginning. We work incredibly well together and are making something that feels significant and important. The button is part of that, but the potential of the site is so much more.

It’s also worth mentioning that if you want to hear Sven and I talk about this story in person, we discussed it on John Sundell’s podcast last year.

Here’s to year two!

Hosting the Swift Package Index

Posted Tuesday, March 30, 2021.
Logo images for the Swift Package Index, MacStadium, and Microsoft Azure

Making any open-source project sustainable for the long term is challenging. You need to find time to work on the code, plan out features, respond to feedback, and if the project is not only code but a live web site, it also needs hosting and regular maintenance.

The Swift Package Index has slightly bigger hosting problems than most web sites, too. In addition to web and database servers, we need a significant amount of processing power to monitor and analyse constant package releases and run our build machines that deal with the ~30,000 Swift builds per month that process through our compatibility build system. 😅

We’re delighted to say that as of today, all of the Swift Package Index’s hosting problems are taken care of thanks to the generosity of two companies that believe in open-source.

First, MacStadium. The site has run on their Mac mini infrastructure since the public launch through their open-source programme. They have accommodated every request we have made, including giving us access to a DTK, which enabled Apple silicon compatibility results months in advance of M1 machines becoming available. They are a pleasure to work with and consistently supportive. Thank you to Heather, Brian, and all of the support team at MacStadium.

However, while running the web hosting, the database, package update monitoring, Linux builders, and all the Apple builders all on the same hardware is possible, it’s not ideal. Our build queue had become quite delayed because we were asking those Mac minis to do too much.

That’s where Microsoft Azure stepped in. We have now eased the load by migrating our web hosting and Linux builders to Azure. The site is load balanced and redundant, which not only increases speed and availability, it also decreases the maintenance effort required. Working with Microsoft has been an absolute pleasure, and their only priority was to see the project succeed. Nothing was too much trouble. Thank you to Stormy, Thomas, Shelby, and Candice.

With the web servers, database, package update monitoring, and Linux builders hosted at Azure and the macOS, macOS ARM, iOS, tvOS, and watchOS builders at MacStadium, the hosting for this project is secure and stable for the foreseeable future. Thank you so much to both companies for their help. Running this site wouldn’t be possible without them.

Note: It’s worth mentioning that this blog post is not a condition of any agreement between us and either of these companies. All this support came without any expectation of anything other than enabling an open-source project. We are writing this purely because we are grateful for their help. ❤️

What’s Next? - February 2021

Posted Tuesday, February 2, 2021.

It’s been about seven months since we launched the Swift Package Index and we’ve been hard at work implementing a few major updates. Most significantly, real-world package compatibility, a comprehensive set of RSS feeds and Twitter updates, and most recently inline README files.

But we’re far from done! 😅

So, in the spirit of letting you all know what we’re planning for this project next, we wanted to start a regular series of blog posts about the direction we’re heading with feature development. We’ll post every couple of months, giving a quick recap of what we’ve achieved along with where we’re heading next. It’s not a formal roadmap. It’s more a declaration of where our focus will be for the next couple of months.

So without further ado, what are we focusing on next?

Intermediate Search Page

Currently, search results are displayed inline on the home page, which makes it hard to share package searches (there’s no URL for a search), or integrate with productivity tools like Alfred or Raycast, or even DuckDuckGo through their bangs feature. We want to fix that.

Package Collections

If you’ve been following the package collection work in the SPM project, then it might have struck you that the Swift Package Index would be a great source of package collections, when they are added. We agree! We’ve already been working hard to prepare for this feature and want to have something publicly available as soon as the feature becomes available in SPM.

Design updates

The site design suited our feature set at launch but has become a little overloaded as we added features. This work has already started, with a new package page layout rolling out last Friday. There’s more to do though, including improvements to the home page. The lists of recent package additions and recent package releases are fine, but we can do better!

The Swift Package Index macOS App

We’ve always wanted to create a Swift Package Index Mac app of some kind. The app’s main purpose would be to allow you to easily try packages out in a Swift Playground (utilising the wonderful Arena tool). But it might end up being more over the long term. We want to define what this app will do, and then get something out that works towards that definition. We talked about the interesting story behind the “Try this package” feature on John Sundell’s podcast last year if you’re interested!

And More…

This is not a comprehensive list of things we’ll do. This is a young project, and we want to keep development agile. If we have a great idea, it may go into the plan ahead of everything you read above!

Thank you!

We’d also like to use this update post to say a huge thank you to our generous sponsors. We wouldn’t be able to put in the time and effort needed by this project without the generous support of Alexandre Colucci, Artem Loenko, Bartosz Dolewski, Boris Bügling, Byron Davies, Chris, Dominik Hauser, Federico Zanetello, Gleb Shevchenko, James Sherlock, Jason Bruder, Jeffrey Macko, Joseph Heck, Joshua Kaunert, Juanpe Catalán, Jules Coynel, Kyle Lee, Leo Picado, Marcos Griselli, Massimo Biolcati, Matěj Kašpar Jirásek, Max Desiatov, Nathan Harris, Oliver Drobnik, PatrikTheDev, Paul Farnell, Paul Hudson, Peter Steinberger, PSPDFKit, Riley Testut, Robin Kunde, Sam Symons, Scott Robbins, Stefan-Cosmin Romanescu, Thomas Dohmke, Tim Condon, Umar Haroon, Vadim Shpakovski, and Xavier Lowmiller. Thank you all! ❤️

Would you like to be part of this list next time we post an update like this? Sponsoring the Swift Package Index is done through GitHub Sponsors, and your support will allow us to continue to build this site into something special.

Inline README Files!

Posted Thursday, December 24, 2020.

If you’ve visited the main package index site over the last week or so, you might have noticed something new. Scroll down a little, and you’ll see the README file for the package, right below all the other information that we aggregate.

When we consider adding a new feature, at the heart of our decision has to be the question “Does this feature allow people to make better decisions about the package they’re considering?” Rather embarrassingly, the README file is probably the most important piece of information when picking a package dependency. Let’s not focus on that though, and instead be thankful that we finally got around to adding it! 😅

We’re aware of a couple of problems with README files that contain embedded HTML or make use of in-page anchor links that are often used to build a table of contents. We’ll get those fixed over time, but decided that it was worth rolling the feature out even with a couple of remaining issues.

Of course, we’ve not been able to test every single README file, so If you notice any other issues, please let us know by opening an issue.

Thank you all for your support this year, and we hope you enjoy this feature that brings our year of development on the Swift Package Index to an end. We can’t wait to show you what we have planned for 2021!

Keeping Up To Date with Swift Packages

Posted Wednesday, November 25, 2020.

How do you keep up with new releases of the packages you use? How do you discover new packages? It’s a tough challenge to keep up to date with everything that the community releases, so we have some announcements today that can help you stay informed!

Before we get started with the announcements, we want to credit James Sherlock for his invaluable contributions to building the Twitter updates feature. He was instrumental in getting this feature up and running and has been making other improvements in several areas of the site since we launched. Thank you, James!

Updates via Twitter

You may already be following the @SwiftPackages account where we tweet occasional updates about the project. But as of yesterday, we’re also tweeting on the @PackageFirehose account. As you might guess by the name, this is a high volume account that posts every time there’s a new package added to the index, and every time there’s a new release of a package that we track. It’s around 20 tweets a day, if that counts as high volume!

A screenshot of a tweet from the package firehose account.

While we’re on the subject of Twitter, we’d love to know what you’d like to see us do with the @SwiftPackages account. We’ve set up a short survey if you have a couple of minutes. We’d love to hear your opinions.

Updates via RSS

We’ve had RSS support for a while now, but we’ve not talked about it on the blog before, so it’s worth mentioning. If you’d prefer to keep up to date via a feed reader, or if you want fine-grained control over what type of package updates you’d like to be notified of then subscribe to one of our RSS feeds:

One advantage that these feeds have over Twitter is that they can be filtered with query string parameters. The package releases feed accepts four parameters; major, minor, patch, and pre. Pass true to any of these parameters to filter the feed on that part of the semantic version number. So, you can subscribe to only major package release, or major and minor package releases combined, or if you only want to know about pre-release versions, you can find those here. Use any permutation of the parameters to get exactly the feed that you want.

We hope you find both of these mechanisms for keeping up to date useful!

The Swift Package Index Metadata File – First Steps

Posted Friday, October 30, 2020.

The vast majority of the information you see in the package index is generated from a single URL, the location of the package’s git repository.

From that URL we gather data from the repository itself, the package manifest, the GitHub API (yes, it’s GitHub only for now, but we do plan to support other hosts), and from running builds of the package to check compatibility with operating systems and Swift versions.

Edge cases

Of course, as with everything in software development, it’s not long before you find yourself talking about edge cases.

While creating the build system, we quickly encountered packages where watchOS targets failed to build. We use Xcode’s automatic scheme creation functionality while running builds, but Xcode generates schemes that always include test targets, and XCTest isn’t available on watchOS. Using automatic schemes for watchOS builds wasn’t going to work.

The only way to have these builds succeed is for our build system to use a specific scheme, rather than an automatically created one. Many packages already have schemes for this purpose, but as package authors can give schemes any name, it’s not possible to easily discover them.

Enter the Swift Package Index configuration file

Rather than trying to keep the build system 100% generic, we needed a mechanism for package authors to specify some configuration information, such as the scheme name to use to build watchOS targets.

We support this via a configuration file, .spi.yml which we will look for in the root of your Swift package repository.

Schemes for watchOS

You can see the popular package swift-composable-architecture by Pointfree making use of this config file:

version: 1
builder:
  configs:
  - platform: watchos
    scheme: ComposableArchitecture_watchOS

Our build servers look for this file and use any scheme information we find for each platform. If a platform isn’t listed, we use our default heuristics to determine the scheme as outlined in our build FAQ.

Images for Linux

Custom schemes aren’t all we support though. Package authors can also use the .spi.yml file to configure custom docker base images for our Linux builds.

We build packages for Linux with docker commands, selecting between the various base images that Apple provide.

/usr/local/bin/docker run --rm -v "$PWD":/host -w /host swiftlang/swift:5.2.4 swift build --enable-test-discovery

This works well unless a package requires an OS-level dependency, like OpenSSL.

We may eventually provide our own set of base images for supported Swift versions that also include common dependencies like OpenSSL. For now, we are asking package authors whose builds fail due to missing dependencies to create their own for the Swift versions they support.

For security reasons, we don’t allow package authors to specify arbitrary images in the configuration file, so if you need to use this feature, please raise an issue asking for us to support a custom base image.

The AWS library Soto is an example of a package that makes use of this feature in its .spi.yml file:

version: 1
builder:
  configs:
  - platform: linux
    swift_version: '5.0'
    image: adamfowlerphoto/aws-sdk-swift:5.0
  - platform: linux
    swift_version: '5.1'
    image: adamfowlerphoto/aws-sdk-swift:5.1
  - platform: linux
    swift_version: '5.2'
    image: adamfowlerphoto/aws-sdk-swift:5.2
  - platform: linux
    swift_version: '5.3'
    image: adamfowlerphoto/aws-sdk-swift:5.3

Future direction

It’s early days for this configuration file. We created it to solve issues which package authors were facing trying to make sure that the language and platform compatibility we report reflects reality.

We expect this file to evolve, which is why it carries a version number. Here are some of the things we are planning to add over time, so that package authors can control them:

More metadata possibilities are under discussion in this GitHub issue. This is a great place to chime in if you have suggestions.

File format

We chose YAML as the file format because it is an uncontroversial, universally beloved format that is entirely free of issues. 😬

Seriously though, we chose YAML as it’s relatively easy for a human to read and write, and it’s well supported in Swift. While it has some downsides compared to JSON, we believe it’s a more practical format given that package authors need to create this file manually.

Of course, we’ll let adoption be the guide as to how well it is working in practice!

Funding the Future of the Swift Package Index

Posted Monday, October 12, 2020.

It’s been almost four months since we launched the Swift Package Index, and we’ve been so happy to see so many positive reactions to it across the community. Thank you all for checking it out and spreading the word.

It’s still early days for Swift Package Manager adoption but we are confident that it will become the dominant technology for Swift dependency management, and we want the Swift Package Index to grow right alongside it.

We believe we’re off to a great start with what we have so far, but this is only the beginning. We have so many ideas for “bug fixes and improvements”! 😂

Many open-source projects are just code and don’t need constant maintenance and monitoring, but the Swift Package Index is a little different. It’s a hosted instance of the site with a reasonably complex automated build system that has run over 600,000 builds so far (increasing by 7,000 more every day!). It takes significant time and effort to keep things running smoothly.

We want to find a way to make sure that we can continue to give this site the time and attention it needs, but we don’t really want to go down the route of including advertising. Instead, we are opening up sponsorship on GitHub.

If you (or your company) would like to support the site, we would be thrilled to be able to make this a community-funded effort for years to come.

A screenshot of our GitHub sponsors goal showing 1% of 100 sponsors.

Also, a huge thanks to Oliver, who beat us to even announcing this by sponsoring the project yesterday! Thanks for being our very first sponsor. 😍

Thank you!
Dave & Sven

Building 3,238 Swift Packages for Apple Silicon

Posted Wednesday, September 9, 2020.

As part of the Swift Package Index build system, we have processed what must be the most extensive test of Apple Silicon compatibility outside of Apple.

At the time of writing, we currently have 12,942 Apple Silicon builds in our database covering 3,238 packages. Why so many? We test up to three versions of each package and compile each version with both xcodebuild and swift build.

I’m sure that many of the Apple Silicon DTK machines Apple shipped around the world are working very hard, testing apps for compatibility, but I’m not sure any of them have worked quite as hard as ours has. 🚀

What did we find?

As you might expect, there’s plenty of good news. For packages with successful Intel builds, the vast majority also successfully build for Apple Silicon. 👍

There are failures, though, and the most interesting failures are where the build fails on Apple Silicon while the corresponding Intel build succeeds. In cases like this, that indicates a compatibility issue that will potentially need some attention from the package author. We’ve identified 139 packages (~4%) where this is the case, and have compiled a list of them for reference.

There are a few common errors that we’re observing in these failures, and most of them are from one of the following problems:

Even though 139 failing packages might sound like a lot, it’s not that bad, especially when you consider that many of the failures all fail due to an issue in a common base dependency, PerfectLib.

Where do we go from here?

Even though the results are generally positive, it does show that there’s a bit of work that the community need to do to ensure the transition to Apple Silicon is as smooth as possible.

If you’re a package author and do not have a DTK machine at your disposal, please feel free to use ours via the Swift Package Index build system! Make changes to your packages, push them to a pre-release tag (for example, X.X.X-arm-beta-X) and the build system will pick up that tag and build it. You should see new build results, usually within a couple of hours of pushing a tag.

If you have any problems with the build system, there’s a couple of ways to contact us. We’re in the #swiftpackageindex channel over at the SwiftPM Slack, or if it looks like there might be something wrong, please raise an issue on GitHub.

Launching Language and Platform Package Compatibility

Posted Thursday, August 20, 2020.

What’s the first question you need an answer to after finding a package that fits your needs?

“Does this package work with the Swift version and platform that my app uses?”

When we initially launched the Swift Package Index, we attempted to answer this question with the metadata available in the package manifest. Namely the swiftLanguageVersions and platforms properties.

The problem is that neither of those properties is perfect. swiftLanguageVersions isn’t granular enough, only officially allowing values of v4, v4_2, and v5. The platforms property is better, but doesn’t let package authors declare compatibility with non-Apple operating systems such as Linux.

Wouldn’t it be fantastic if you could see a matrix like this for every package? 😍

The language and platform compatibility matrix for PromiseKit.

Look at how information-rich that matrix is. You can instantly see that the latest stable version of PromiseKit is compatible with every version of Swift back to 4.2, and every platform, including Linux. Then, you can see that the alpha version in development drops support for iOS, tvOS, and watchOS, and Swift 4.2. That seems suspicious, right? Keep looking, and you’ll see that the default branch fixes all those issues and restores compatibility. I’m confident looking at that matrix that when 7.0.0 is released, it’ll have green ticks across the board, but I also know to not depend on this current alpha. That’s practical, actionable information.

When we started thinking about how best to solve this problem, the obvious best solution was to build the packages! What better way to see if a package is compatible with Swift 4.2 than to build it with the version of xcodebuild that shipped with Xcode 10.1.

So that’s what we did, and it’s available right now. Why not give it a try by searching for a few of your favourite packages? 🚀

Accurate, real-world compatibility data

It’s a little more complicated than “just build each package” though. A package might build with Swift 5.2 on iOS, but that same build might fail using Swift 5.2 on macOS due to a UIKit dependency, or other macOS specific issue. What’s needed is a matrix of builds to generate an accurate picture of compatibility.

So, if we run builds using Swift 5.1 on iOS, macOS, tvOS, watchOS, and with Linux and any of them pass, it’s compatible with Swift 5.2. If any version of Swift builds without failure on iOS, then the package supports iOS.

We ended up with a platform list of:

We then decided on a list of Swift compiler versions we’d like to check compatibility with:

That’s up to 32 builds per package, but that’s just the beginning. What if there’s a stable release and a beta release? The stable version might support Swift 4.2 and higher, and the new beta might drop support for anything less than Swift 5.2. That’s information which would be important when choosing a package, so we need to show it. As we also track the status of the default branch, we must build that too, and we’ve quickly arrived at a place where we might need to run 96 builds per package! With almost 3,200 packages in the index, that’s potentially more than 300,000 builds! 😅

In practice, it’s less than that as most packages don’t have a current beta release, but it’s still a lot of builds. We’ve processed more than 200,000 builds as I write this, and we’re not quite finished. As of today, we’re at 99% though, so we almost made it before launch! 😬

If you’ve been following these tweets, it should be obvious what all that processing was! Let’s take a look at the last 30 days of CPU graphs for our production server, a 2018 Mac mini with 32Gb RAM and a 6-core i7 CPU:

A graph showing a few spikes of CPU activity, followed by a sustained 100% CPU load.

You can see a few of our final test runs in that graph, and then we started processing for real. Since then, we’ve kept the CPU completely pegged for more than two weeks. We’ve also had our staging Mac mini, a spare 2016 MacBook Pro, and a DTK working on builds too.

Everyone loves badges

Providing compatibility information on this site is one thing, but everyone loves adorning their packages pages with shields.io badges, don’t they? If you maintain an open-source project, wouldn’t you like to show off real compatibility status in your README file, like this?

A screenshot of a GitHub page with badges that show the Swift and platform compatibility for the package.

If you’re a package author, click the “Copy badge” button below each of the compatibility matrices and you’ll have a Markdown image link in your clipboard, ready to use.

Your users will always see live, accurate compatibility information that updates whenever you release a new version.

Credit where it’s due!

First of all, we’d like to thank our kind friends at MacStadium for providing the significant hosting resources for this project as part of their open-source programme. At one point we were a little concerned that we might melt their machines, and we very glad that we didn’t. They’ve performed incredibly.

We also want to say thank you to Ankit Aggarwal and Boris Bügling of Apple. Their tireless help and support on the SwiftPM Slack saved us countless hours figuring out the correct way to approach this problem.

Finally, we’d love to say thank you to everyone who provided help and feedback along the way as we built this feature. We couldn’t have done it without any of you.

Wrapping up

Some package authors set up continuous integration for their packages and of course, that includes a build step. That usually only covers one version of Swift though, and the information gets hidden away in a different place in each repo. We think that by centralising this data and making it available for all packages, we should be able to help this community make better decisions about their dependencies, and that’s what this site is all about.

We hope you love this feature as much as we do! ❤️

Say Hello to the Swift Package Index Blog

Posted Sunday, August 16, 2020.

Hello! 👋

It’s always hard to know when to ship v1 of a project. Which features get included? Are there things that can wait? It’s a balancing act. You’re excited to show the world what you built, but is it good enough?

One decision we made when shipping the Swift Package Index was that a blog could wait. Looking back now, it feels like that was the right decision. The launch went smoothly, you all reacted very kindly and positively to it, and we went back to work fixing bugs and working on new features.

However, we always knew we’d need somewhere to write about what we are doing with this project. We’d love to tell you about some of the interesting technical challenges we faced, but most of all we want to tell you about the new features we’ve been building...

What features? You’ll have to wait just one more week to find out what we’ve been working on, but we’re pretty excited about it, and that’s why now was the right time to launch this blog.

Now we just need to keep writing on it! 😬

The Swift Package Index is a search engine for Swift packages with one goal. To help you make better decisions about dependencies. For more information, the best place to start is with the Frequently Asked Questions.