Success Stories: How One Man’s Race to Beat A Rocket Launch Enabled Thousands of Linux Developers

August 15, 2023

Sponsored Story

At any given moment, seven scientists, engineers, and pilots are floating 250 miles above our heads. They are the crew of the International Space Station, and they are orbiting the planet at 17,500 mph.

Except for the limited gravity, everyday life on the ISS isn’t terribly different from life on Earth. Astronauts conduct experiments, maintain the spacecraft, exercise on stationary bikes and treadmills, watch movies, read books, and call their families. They also build stuff, albeit with a bit more ingenuity than back at home.

In-Space Servicing, Assembly, and Manufacturing (ISAM) is an emerging capability for the in-orbit construction of space vehicles and large satellites. The concept is built around a robotic connector called the intelligent Space System Interface (iSSI), a common interface bus that physically attaches modular payloads to a space platform called the Orbital Logistics Vehicle while also transmitting up to 5 kW of power, 1000BASE-T Ethernet and medium data rate CAN bus serial signals, and thermal energy between them.

The iSSI hardware is being qualified for spaceflight to NASA’s Technology Readiness Level 7 (TRL 7), which can only be accomplished by testing the equipment in its intended deployment environment – outside the Japanese ISS experiment module “Kibō”. During the tests, the iSSI team also flight-qualified the Microchip PolarFire SoC FPGA used to control the robotic connector and performed an experiment to characterize the radiation environment the electronics were subjected to.

But this wasn’t the first time the iSSI technology stack had been put through its paces. That began on Earth with embedded engineers.

Racing a Rocket                  

Daire McNamara wears a lot of hats. He’s a director, project manager, and firmware developer at Emdalo Technologies, an embedded software development firm based outside of Limerick, Ireland, that specializes in a range of engineering disciplines, including chip bringup.

“One of the key jobs in that is, as the chip comes back from manufacture, does it work?” McNamara explains. “Does it do what it’s supposed to? Do the boot ROMs, these early pieces of code in the chip that bring it to life, work? And if they don’t work, is there anything that you can do to get in there to fix it?”

When the iSSI project got underway, the PolarFire SoC FPGA lead partner Skycorp had selected as one of the system’s primary control platforms had only just hit the market. As a Linux-capable processor built around a RISC-V CPU cluster, tens of thousands of FPGA logic elements, and an advanced memory protection scheme, the PolarFire device checked a lot of boxes for the mission. That said, the technology infrastructure around the processors was still maturing.

“At the time, several of the hardware capabilities of the PolarFire SoC were not exposed through the software development kit,” McNamara said. “But from Skycorp's point of view, the rocket was leaving when the rocket was leaving. There was no negotiation on the timeline.”

“Obviously for the International Space Station, that’s a highly reliable environment and they needed several custom extensions to improve the reliability of it,” he continued. “So, at that point, Skycorp engaged directly with us as we had been heavily involved in the software development kit anyway for Polar Fire to add those custom extensions as a design service early for them.”

Dennis Wingo is CTO at Skycorp, an ISAM pioneer, and it’s his company’s Orbital Logistics Vehicle that will use the iSSI as an interconnect for hosting custom payloads. Over some 40 years in the aerospace and electronics industries he’s developed technologies that have flown on all sorts of space systems, from nuclear missiles to the space shuttle to, of course, the international space station.

So, from a technology adoption standpoint, Wingo has experience with the things that work in the aerospace electronics industry and those that don’t.

“The ISSI, we call it the USB of space,” he said. “And so that enables us, with a modular standardized connector, just like what we've done in the computer industry, to be able to lower the costs and build large structures in space.

“One of the things that have held space systems back a lot is that traditional aerospace methods are such that everything is done from scratch, the development tools suck, the standard processor for most space applications is still a PowerPC that I first qualified in the early 1990s … they're still using that today with terrible operating systems and things like that,” he continued.

“If I can get something like the Microchip PolarFire, which has a crucial element that none of the other processors have – error correction at the register level inside of the ASIC side of the SoC and also in the FPGA side – and be able to run a modern operating system like Yocto Linux, that makes for a very capable system,” Wingo said.

“The biggest problem we had is that, at the time we started this, which was early in 2020, the PolarFire SoC ecosystem was immature,” he went on. “A lot of the capabilities of the SoC, the drivers, the firmware, the things that make the damn thing work, either were not written or didn't work right.

Looking for assistance, Wingo and his team at Skycorp turned to Microchip, who directed the aerospace engineers to Daire McNamara who “started knocking down the bugs,” Wingo added.

Staying Connected

In a past life, Cyril Jean, Associate Director of IP Engineering at Microchip Technology worked with McNamara at a Bluetooth IP company. Now the two are partners in the Microchip Mi-V ecosystem, where Emdalo was heavily involved with the initial bringup of the PolarFire SoC, helping with the bootloader and original Linux port and then later during performance optimization of the hardware and software stack.

When Skycorp came to Microchip for help, the candidate to fill the void was obvious.

“As Microchip, we can't do somebody’s product. That's not really part of what role is, but we do have people who we work with, who can do that,” says Jean. “So, he was there from the bootloader to getting to a Linux port. That's where he was heavily involved, on that initial bring up of the system. And then after that it was how we could improve performance on part of the system, where he had to really understand both sides of the house, the software on silicon side.

“Having worked together, we know each other, what we can do, what we're good at, how to work with each other,” Cyril says. “[Daire’s] already there and he's familiar with all of those blocks. So that was an easy one to match. Plus, he was very excited about working about something on the ISS.”

The Internet of Space Things

Now, back to McNamara.

“We added keep alive messages to the PolarFire, which enabled Skycorp to detect if there was any problem with any of the subsystems in the experiment and reboot them to restart them if needed. And they needed the in-orbit upgrade capability – the ability to upgrade all parts of the design – including the FPGA design underpinning the software, the Linux software that was running on the PolarFire, and even the boot ROM. We added all those capabilities to the PolarFire SoC.”

In all, the work that Daire and his team completed enabled Skycorp and its iSSI partners to capitalize on an architecture they couldn’t have before: the Internet of Space Things.

“On the space station they have what's called the Joint Station LAN, which is an Ethernet-based local-area network,” Wingo explained. “They encapsulate IP packets in a NASA protocol and ship them up to geosynchronous orbit, back to the ground, and through all these VPNs through NASA.

“So, we were able to use a standard web browser to connect all the way through the VPNs and everything to the payload, through proxy servers and all that, and operate the entire experiment through a web server on our payload,” he said. “We were able to operate the experiment from our office and actually had the world's first web server on our payload running Yocto Linux on the PolarFire, so we were able to use all modern tools to do this in a very cost-effective manner.”

Engineering Needs its Heroes

The iSSI platform successfully completed flight qualification earlier this year and was returned to Earth in a SpaceX Dragon capsule. It’s being sent to a lab for analysis, after which Skycorp and its partners will reveal their findings in IEEE journals, likely by the end of the year.

For McNamara, the experience of working on a system that deployed on the ISS was “incredibly uplifting,” and gave he and his team the opportunity to tune in live and see their work being launched into space on a Northrop Grumman NG-17 craft, dock at the ISS, and be packaged for deployment in the vacuum outside of the Kibō module.

“We've monitored its behavior, how it's working and it's worked really well all year,” McNamara glowed. “We're really looking forward to seeing the kit, to see how it survived its year in space.”

But just because the flight qualification portion of the iSSI project is complete doesn’t mean the impact of McNamara’s work Daire on the PolarFire SoC has diminished. In fact, just the opposite.

Wingo explained:

“We have individual stars, Daire is a star, and there are lots of folks like Daire out there and these types of activities that lower the barriers to entry to people really help us all.

“There were some bugs that Daire and the team helped us work through and became part of a larger ecosystem because they get propagated to GitHub to the next release of Yocto,” Wingo continued. “Now the Yocto or the kernel, the common kernel, is leading to the PolarFire being part of the Ubuntu ecosystem as well.

“And so, all these things that we do and that Daire is doing are helping a wider ecosystem,” he adds. “And I think this is an important thing for all embedded engineers, that an individual can have a much, much wider impact no matter what you're working on if you're part of a community, you're part of a team where you can make this stuff happen.”

Next up for Daire, the team at Emdalo, and their partners at Skycorp? Currently they’re working on deploying the world’s first data center on the moon.

Seriously.