REV-1

Status: Concluded

Introduction

On 10 June 2023, AggieSat Laboratory’s REV-1 team competed in the international CanSat Competition in Blacksburg, VA. The REV-1 system was delivered on time and under budget. However, during flight, the probe did not separate from the canister. There were many issues that caused REV-1 to fail. The team determined that the main system error was within the flight software. The software subteam had major personnel issues that led to this failure. Additionally, lack of system-level testing proved to be fatal for the physical mechanisms of the system interacting with the software. At a team level, time management and subsystem integration were the two largest contributors to failure. A lack of well-written documentation was also partly to blame. The following explores the issues and successes of the REV-1 submission to the CanSat Competition 2023.

Purpose

The purpose of this page is to provide analysis of the REV-1 performance. This page shall include any and all failure points in the project, lessons learned by the members, and advice for current and future CanSat projects. Note that parts of this page are written from the perspective of the Project Manager, Elena Welch.

Team Organization

The REV-1 team was organized into three subteams as show in the figure below.

Launch Day Sequence of Events

On 10 June 2023, hours before launch, the REV-1 system was power cycling. Current to the microcontroller dropped too low when all of the servo motors were activated, which forced the system to reboot. Upon launch, all the team could hope for was a successful radio communication link and egress of the probe from the canister. The aerobrake servos were unplugged as an attempt to mitigate the current droop the system was experiencing. That would hopefully allow the microcontroller to stay powered without rebooting, the team could at least receive telemetry data, and the probe may fall safely to the ground under the larger parachute. But after the team placed the REV-1 system on the launch pad, the radio link dropped, and no telemetry was received. During flight, the egress hook never unlocked, so the probe never exited the canister. The REV-1 system, canister and probe together, fell hard under its one-foot-diameter parachute, breaking most of the onboard mechanical components when it landed.

Software Subteam Perspective

The software side of the project had several problems, both internal to the subteam and external. Externally, the structures (STR/DEPL) and the electronics (EPS) team did not finalize their hardware design and selection until early June, and both subteams provided very few prototypes. The flight computer board was not completed until hours before the competition, and a breadboard prototype was never received by the software subteam (SOFT). STR/DEPL tweaked the release and deployment mechanisms until very late in the project, so a hardware prototype was never available for testing. Internally, SOFT struggled with personnel issues – the subteam was made up of two people, and the SOFT lead (one of those two people) was absent for the latter portion of the project. The flight software was thus written almost entirely by one person, which severely limited the subteam’s bandwidth, especially considering the person who was responsible for the flight software also spent a lot of time working on things for the other subteams. Additionally, the majority of the software was written in the days leading up to the competition. This severely negatively affected the quality of the flight software, and this was further worsened by the changes still being made to the system as the software was being written. Similar problems applied to the ground station software, and members of other subteams had to step in to complete that portion of code.

The above problems translated into several software issues which directly caused the failure of the system: confounding of the system’s state, and buggy/nonfunctional hardware drivers. This information is documented in the REV-1 repository on the AggieSat Laboratory GitHub.

To resolve and prevent these issues, several measures should be taken. First, the hardware should be finalized early, and subsystem- and system-level prototypes should be made available throughout the development process, even if not every component or mechanism is implemented yet. Second, SOFT should be assigned more members. The two that were assigned were insufficient to create the software, and one of the two members wrote virtually no code. Third, the software should be started much, much earlier in the project. While a basic software development plan was conceived early on for REV-1, it was not executed properly due to a lack of personnel availability. And fourth: while hardware availability is very helpful for software development, following good software engineering practices can shorten the time needed after hardware is built. Software engineering practices that would have helped the SOFT team work more efficiently include function stubbing, use of test suites, better abstraction, and use of a software development plan, which are explained below.

Function stubbing is the practice of creating empty function declarations and definitions for the purpose of defining how different parts of a software will work together. This is useful because it allows non-implemented parts of the software to be abstracted into an Application Programming Interface (API) that can be interacted with by the rest of the program, allowing a software developer to write software around blocks and slot the implementations in later. Function stubbing would have helped SOFT by allowing the subteam to write the control parts of the software (state machine, altitude calculations, and simulation mode) without having to implement the hardware interface portions of the software such as device drivers. This would have allowed the software to be working before any hardware was created at all, leaving only the implementations of device drivers and debugging to be done after the hardware was completed.

Test suites are a software engineering paradigm by which portions of a software can be tested against various situations. They often take the form of automated scripts built into development environments that provide inputs for a program and monitor the output for a desired result. Test suites would have helped SOFT by allowing the subteam to leverage simulation mode (a required part of the software) to execute full dry runs of the software at all points during development. A good test suite would implement a one-click simulation of the entire flight from launch to recovery. When combined with function stubs, this paradigm would allow for full-stack testing, no matter the state of the hardware.

Abstraction is the software engineering practice of hiding complex systems behind simple interfaces. It is extremely powerful for embedded systems because it allows a software developer to interact with hardware and software components through simple functions. This greatly simplifies program flow and control and is imperative for readable and collaborative software. A large issue with the REV-1 software was the complex and inefficient implementation of the system’s state machine. Abstraction of the state machine would have greatly reduced the state management problems the system faced. At the time of flight, the state of the system was stored in several different variables and accessed and mutated by several different functions in the code. Abstraction and proper encapsulation would have provided a single, unified way to interact with the state of the system, which would have made state management far simpler.

The use of a software development plan is another key piece of the software puzzle. A software development plan is a tool that lays out what individual pieces of the software need to be developed, how the software will be developed, and who will develop it. A common method to do this is via a mind map, which is a tree-shaped diagram that successively divides each part of a program into smaller, simpler pieces. These pieces can then be assigned to different members of the subteam, put onto a Gantt chart, and tracked/organized however project management sees fit. A software development plan is the most important part of software design because it allows for the software paradigms listed above to be planned from the start and lays the foundation for good software to be written.

Overall, the software subteam suffered many issues, both external and internal. However, with proper software engineering procedures and more subteam members, the internal issues can be addressed, and the external issues can be mitigated.

Electrical Subteam Perspective

The Electrical Power and Radio Frequency (EPS) subteam originally planned to design and procure a Printed Circuit Board (PCB) to connect the various electrical components of the REV-1 system. It cannot be stressed enough how important it is to use a PCB to connect components together. Using a PCB allows for easier debugging, no wire snagging, less mass, and a smaller footprint compared to implementation of a breadboard or protoboard. While a PCB for REV-1 was designed as shown in the figure below, it would have been too large to fit in its designated area. Time had to be taken to reduce the PCB size to fit inside the probe, and the individual who took responsibility for resizing the PCB was also trying to figure out the camera functionality (discussed later in this section). By the time the updated PCB was ready to be ordered, it would not have arrived in time for system integration and travel to Virginia. Even when ordering from an American manufacturer, the PCB likely would not have arrived in time, and the price would have been almost five times higher than ordering from China.

Since getting a PCB was out of the question, the EPS subteam chose to use a protoboard instead. As such, they had to hand-solder all the components to the board, and the REV-1 team experienced difficulties debugging the integrated system. During flight, the flight software did not run because the microcontroller experienced a brownout. Brownouts occur when there is a high load to which the voltage regulator can not supply enough current. A voltage drop occurs at the output of the regulator because of the excessive current draw. The Teensy microcontroller was connected to the same voltage regulator output as the servos, so when the servos turned on and tried to draw a lot of current from the regular, the microcontroller automatically turned off. It then turned back on once the regulator was capable of stepping the voltage back up to its nominal value. Hence, the system was power cycling. The REV-1 team tried to remedy this issue by unplugging the aerobrake servos, hoping that there would be enough current for the Teensy during flight. However, there were still three servos plugged in, and the brown out and power cycle issue occurred anyway. The permanent remedy for this issue would be to separate the Teensy’s power source from the servos’ power source, which would be the voltage regulators. EPS did originally plan to use two 5-Volt regulators to supply power to the Teensy microcontroller and the servos separately. However, due to miscommunication between subteam members, this was not done. Wiring became confusing as multiple people worked on the protoboard, and the time crunch of doing this work just hours before flight led to a subpar soldering job, as pictured in the following figure. Future teams must ensure that power-sensitive components such as sensors, microcontrollers, and/or microcomputers are electrically isolated from other components that draw large currents. Teams must also verify that the power drawn from all components does not exceed the maximum power that the regulator can supply.

The CanSat Competition 2023 required the REV-1 system to have at least one camera on the probe and recommended having a second, bonus camera in the canister. The team initially selected the Adafruit TTL Serial JPEG Camera as the primary component. However, during the Preliminary Design Review (PDR), a competition judge advised the team not to use that camera because he had personally experienced issues getting a steady framerate from it. The team heeded his advice and decided to go with the Adafruit 3202 Mini Spy Camera, a component with which they knew previous CanSat teams had success. However, keeping in mind the limited budget, EPS decided to only order one 3202 camera for testing before finalizing its selection. Once the part came in, they attempted to test it, but they struggled to command the camera to begin recording. The team held off on ordering a second camera throughout this troubleshooting period. EPS finally figured out how to operate the 3202 thanks to a forum post with a script that instructed the camera to start recording; this script is included in Appendix A. Unfortunately, by then, all vendors were out of stock of the 3202, so the team had to find a different, third model of camera to use as either the primary or bonus camera. They selected the OV7670 because it was the cheapest, most available option considering the very real time constraints they were facing at that point. The OV7670 was capable of 480p video quality, and although its framerate would be terrible, it met the competition requirements for the probe camera. Additionally, the 3202 had a smaller form factor, so EPS decided to make that the bonus camera on the canister and use the OV7670 as the probe camera. Figuring out how the OV7670 works took them a very long time, resulting in delays to the previously discussed PCB work. Ultimately, trying to juggle the PCB redesign and two cameras caught up to EPS, especially since one member was absent and another was remote for internships. Both the 3202 and the OV7670 were scrapped from the final REV-1 design so that EPS could focus on more pressing matters like finalizing and soldering the protoboard.

Since EPS did no sub-system level testing of their hardware, SOFT had nothing to test their code on. The components were tested individually on a breadboard with Arduinos or other microcontrollers to verify functionality. While this is a perfectly acceptable way to test components, the next step would be to add components one by one and test the code incrementally. This would make debugging easier, and it could be done on a breadboard with the tools already supplied by AggieSat Laboratory.

Another issue arose because the EPS subteam did not document or make schematics of their design process. The easiest way to keep track of design changes would be to make an initial schematic of the pins of each chosen component and document how they are connected in an engineering drawing. The primary reason for making these schematics is so that anyone on the team can wire the components together before the arrival of a PCB. Design changes can then be documented as revisions to the schematic drawings, with proper design change requests and notifications implemented in accordance with updated AggieSat Laboratory standards and practices. The soldering issues above were in part a result of the lack of proper EPS documentation. The schematic shown in the figure below does not meet the standards that AggieSat Laboratory teams should follow. It is unclear and confusing for the average engineering student to decipher. Electronics diagrams and schematics have a long-established, universal format to which the lab should adhere. A clean engineering drawing of each of the components with their pinouts listed should be included in an Interface Control Document (ICD).

Choosing components for the overall system also comes with its own considerations. It is permissible to purchase sensors integrated onto PCBs from companies such as Adafruit. On the other hand, space and mass can be saved by purchasing the sensors individually from Mouser, DigiKey, or other reputable sources, and soldering them onto a PCB in-house. The BNO-055 that Adafruit sells, for example, is integrated onto a breakout PCB with its own voltage regulator. This is the case for every sensor one may buy, as they are intended to be able to work using only the components on the breakout board and a master or peer device to communicate with. To save space in the CanSat, it is probably easier to solder individual sensors onto the PCB.

Components should be tested as soon as they arrive. This should include tests to verify accuracy of sensors. These unit-level tests can be completed on a breadboard with a generic microcontroller; they do not have to be conducted with the final microcontroller and software. For many, if not all, sensors used in the competition, libraries are available for the Arduino framework that demonstrate functionality of the sensors. Tests should be written to verify the accuracy of the sensors to the degree listed in the requirements for the system. These tests should be designed to be as simple as possible, and the instructions should be clear enough that any member can implement them with little guidance.

Structures Subteam Perspective

The REV-1 system had five mechanical stages: the one-foot-diameter parachute deployment, the egress hook, the aerobraking/self-righting mechanism, the three-foot-diameter parachute deployment mechanism, and the flag raise mechanism. The first parachute was the only stage to operate as planned during flight because it was designed to open passively upon egress from the rocket.

The egress hook for the second parachute worked well in testing, however, the hook was never actuated in flight due to a combination of programming and electrical issues. This caused the entire CanSat to fall under the small one-foot parachute all the way to the ground with the probe still trapped in the canister. Given that failure, the aerobrakes would not have been able to deploy since the probe was still inside the canister. Regardless, the servos had to be unplugged for electrical reasons, so the team knew the aerobrakes would not deploy in flight. With those servos unplugged, the self-righting mechanism would also not deploy.

The three-foot parachute mechanism was in development right up to flight time. The parachute supplier gave measurements for the packing volume that did not fit within the CanSat. An additional storage compartment had to be fabricated and attached to the top of the probe at the last minute. The new design is detailed in the figure below. Even with all the last minute changes, preliminary testing showed that the alterations worked well. The flag raise mechanism was promising, but was never fully tested after assembly due to time constraints. During the pre-flight inspection, one of the judges commented that it was very similar to a current NASA project he was working on. Despite that vote of confidence, the REV-1 team decided that the unproven mechanism had to be removed to save weight and avoid any potential collisions within the probe or the electronics.

The structures and deployables (STR/DEPL) subteam succeeded in delivering a physical prototype that met design requirements on time and under budget. However, the lack of a prototype until the last minute gave the SOFT and EPS subteams insufficient time to integrate and test electronics and software. The primary reason for the delay of the prototype was the inability to begin 3D printing the selected material of nylon-carbon fiber until spring classes had already ended. This bottleneck was the result of the team not ordering the filament dryer needed to print with this material until late into the project timeline. The team printed one PLA version on the probe for size reference and testing, but this was not a full-sized model.

Another major cause for delay was insufficient CAD management. The subteam chose to use SolidWorks to create its CAD and adopted a procedure where the most up-to-date CAD model was kept on one member’s computer and periodically shared with the team by uploading files to Google Drive as requested. While this procedure was fairly straightforward and practical for the initial design, it made it difficult for other team members to edit the CAD in its most up-to-date state, and it was nearly impossible for two people to work on the CAD simultaneously. Additionally, this meant that the most current version of the CAD was not accessible to every member at any given time. Due to both the lack of shared responsibilities and access, one team member was left to control all CAD for 3D printing and documentation. This was an inefficient way to handle this challenge. A better form of CAD management must be adopted.

The STR/DEPL subteam did not utilize its four members effectively. There were times when either some or all members of the subteam could not make meaningful progress on the project because of bottlenecks with physical parts, lack of access to CAD, or poor time management. While structures and deployables are critical to the CanSat competition, personnel were more needed on the other subteams. It would have been a more efficient and valuable use of those members’ time to be assisting other subteams during the bottleneck periods. When forming subteams, consider making STR/DEPL smaller. It may also be beneficial to allow more flexible “floater” positions between subteams so that support can be allocated where necessary.

The STR/DEPL subteam delivered a working design, and the mechanisms designed by the subteam worked well. For instance, the egress hook that kept the probe in the canister worked well: the piece was designed carefully and machined well, and if the servo had been actuated in flight, the canister would have released the probe. Additionally, the design of the probe was well-thought-out and versatile. STR/DEPL members were able to add and remove pieces when needed without affecting other subsystems. The aerobraking mechanism only required one design change in response to the canister. Overall, STR/DEPL did well designing the CanSat, but while the probe and canister met all requirements, they lacked ease of maintenance. Outside of a few screws used in servo brackets, all pieces were glued into place. This led to significant difficulties before and at the competition when the team needed to disassemble the aerobraking mechanism and reinstall it. When designing the mechanical components, consider using fasteners such as screws with nuts or heat-set inserts to build robust and easily repairable parts. Additionally, exotic materials should be avoided unless there is an abundance of both experience and time. For example, the 3D-printed carbon fiber nylon filament REV-1 used was strong and flexible, but it was expensive and slowed down prototyping. Consider prototyping in PLA or ABS plastic, even if using a different filament for the final design, to begin prototyping right away. Select materials and hardware that meet strength requirements but are also easy and timely to purchase and work with.

To operate successfully in the CanSat competition, several changes should be made to the way the STR/DEPL subteam operates. First, all required parts and tools should be ordered earlier with the goal of finishing a working prototype by Critical Design Review (CDR). This will allow time for other subteams to use the physical CanSat when developing and testing their components and software, resulting in less guesswork and less room for error. This also allows for a full system test, which would have been extremely helpful in finding failure points in the REV-1 design. Furthermore, having a full working test bed by CDR will allow all subteams to test their equipment in an earlier stage. Second, the team should select and become familiar with some program to share and collaborate on CAD more efficiently so that all members of the subteam can edit the CAD. This will improve the speed at which the design can be created, lead to a more collaborative team environment, and aid in proper time management of members. The REV-1 team suggests using Git because it would be the easiest to implement in terms of maintaining a large repository with many files and contributors. Third, designing should always be done with assembly and operations in mind. Specifically prioritize ease of assembly and disassembly. This will aid in troubleshooting when something breaks and help with any inspection or verifications the team needs to complete. Finally, the team should make an effort to spread the workload out evenly over time and across the team. Better allocation of people’s time in the early stages of the competition can significantly reduce or eliminate many of the challenges the REV-1 team faced.

The competition was a worthwhile experience, but not a major success. Pre-flight and post-flight pictures of the REV-1 system are shown in the following figures. There are a few key takeaways from the structures and deployables experience. CAD management is critical to the success of the team. The most up-to-date CAD models should be accessible to everyone, and design decisions should be shared across the team and well documented to aid in systems integration. Next, it is important to efficiently allocate all the team’s resources. With a limited number of people on the team, the subteams should be split up based on expected workload, and everyone should have an impactful task to work on. Another takeaway is to use materials that are reasonably cheap and easy to work with. Expensive and exotic materials are not necessarily the best to spend team money or time on. Teams should purchase realistic tools, materials, and parts as soon as they know they will be used in the final design (or even just the prototype). AggieSat Laboratory can help future teams succeed by keeping an inventory of sensors, microcontrollers/microcomputers, and materials for testing. The STR/DEPL subteam should produce a working prototype of the CanSat as soon as possible, preferably by CDR. This will be the testbed to prove all the design concepts and allow other subteams testing opportunities as well. Finally, design the CanSat for manufacturing and assembly. Avoid permanently attaching any component that may need to be replaced, repaired, or even temporarily moved for access to other components.

Pre-flight photos of the REV-1 system:

Post-flight photos of the REV-1 system:

Project Manager Perspective

There were quite a few personnel issues on the REV-1 team. The STR/DEPL subteam was too large. SOFT only had two people; they needed three. Beyond the subteam arrangements, there were quite a few members who did not participate after May. Most internships start in June, and I was expecting many of my members to come back after visiting home for a week or so to recharge after finals. Instead, there were a lot of family vacations taken, which surprised me. I also expected my members to request time off or later start dates for their internships/jobs so they could come to the competition, but we ended up with only half the team going. I regret that the entire team couldn’t go because the members that did go gained immense engineering experience, and they were motivated to go again in 2024. Specifically, I wish more underclassmen had gone, because sending them to competition a second time would give us a huge advantage.

To solve the personnel issues, the next project manager should choose their own team, subteams, and leads. In our case, the program manager at the time selected all 10 members, established the subteams, and appointed the leads; I didn’t make any initial decisions about who was on my team. It doesn’t seem like an issue until the project manager has to communicate with the subteam leads. Ideally, the subteam leads should be someone the project manager has worked with before. If I were to do it again, I would have appointed myself to the software subteam. There is a lot to manage, but I think that the project manager should have time to be on a subteam, preferably electrical or software, as those were the subteams that needed the most help.

The software subteam needs considerable restructuring. The best way I can think to start knocking requirements out of the way is pushing working versions of the software at agreed-upon milestones. This means that the subteam can start with a minimal working version of the software that, for example, simply connects the radios together and sends a single message. In the next iteration, they can implement the servos and add the sensors one at a time to pinpoint bugs easily. With a clear-cut setup of what the goals are and when deadlines are, the software subteam could have created at least a working version of the software by flight, even if it left out some of the minor features. These working versions should be detailed in a Software Development Plan, which the REV-1 team started but did not fully implement.

There are skills that AggieSat Laboratory should be providing to its members that it currently does not. For example, soldering is not taught to every major. Since lab membership consists of students from many different disciplines, it should be known that not everyone will know how to solder. This puts undue stress on the students who do know how to solder, as they may end up doing the entire share of the electrical hardware work. Members also may not know how to code or how to document their code. The lack of software planning and documentation in the early stages of the REV-1 project resulted in the delay of product delivery. Additionally, as the code was being worked on, there was no version control implemented, so other subteams had trouble helping SOFT when they needed assistance. There are a number of workshops that should be made available to lab members. I suggest offering, once a semester: basic Arduino/microcontroller usage, 3D printing, soldering, and how to use version control (Git).

Document Management

A major point of failure for REV-1 was the lack of documentation. The competition requires a very small number of documents:

Requirements,
ConOps,
PDR presentation,
CDR presentation,
Monetary budget,
Gantt chart,
Environmental testing procedures, and
PFR presentation.

These documents are the bare minimum for the CanSat Competition. They are simply the ones that need to be turned in to the competition judges. There are many additional documents that should be reviewed by management as proof of progress. While the Gantt chart is required, it seems the scope of it is too large for this project, and using that for planning is daunting. I think it’s easier to use a Work Breakdown Structure and a management-imposed Go/No-Go schedule. If, by any given milestone, the team cannot meet the go condition, they should be required to pull out of the competition.

Specific documents that shall be written by the team in addition to those required by the competition are:

A software development plan,
A CAD version control plan,
An Interface Control Document (ICD) for connections between components, and
A postmortem report.

The CanSat team should also keep a log of the documents created and maintained, with the file path specified, so everyone can find what they need.

AggieSat Laboratory management should now have a better idea of the documentation skills members are lacking in. Proper version control, software documentation techniques, and design changes are areas that the lab must standardize and implement across all projects. Many mistakes arise from one person making a decision and neglecting to document why or how the change was made. Sometimes, there needs to be approval from management before decisions are made. Additionally, the documentation schedule needs to be implemented by management in a way that feels serious to the team members working on the project. Without consequences, the documentation process will not be taken seriously and will negatively affect the lab as a whole.

Conclusion

Overall, the challenge that CanSat offers is a great opportunity to learn the engineering design process and expose students to different engineering fields. There is room for young engineering majors to learn about version control, documentation, and how design decisions affect the overall system. The trip to Virginia was motivational for the REV-1 team, despite the stress and lack of sleep. If the contents of this document are implemented seriously, AggieSat Laboratory will make a strong case for itself at the 2024 competition. For our first competition, the team did wonderful work. Everyone did an amazing job of keeping their spirits high even when things went wrong. We were extremely blessed to make the competition this year, and I hope future students will experience an entire design lifecycle such as the one CanSat provides.

This competition could not have been attended without the support of AggieSat Laboratory management and sponsors. The REV-1 team would like to thank Dr. Helen Reed for sponsoring our trip and allowing us the opportunity to make mistakes and learn from them. A big thank-you goes out to Marco Peredo, who greenlighted the project and gave us momentum to enter and compete in the competition. We would also like to thank Alexander Duffy for working with the university to get us on the plane to Virginia, and for being a friend that we all needed during the final days of the competition. The contributions of these people cannot be overstated. To everyone on the team: great work! We truly hope that AggieSat Laboratory can build off what REV-1 did this year and take it with them into 2024 and beyond.

Appendix A – 3202 Camera Arduino C++ Script

    void setup() {
  pinMode(9,OUTPUT); //Set camera pin

// Hold HIGH and trigger quick (<250ms) LOW to take a photo. Holding LOW and trigger HIGH starts/stops video recording
  digitalWrite(9,LOW);
  delay(1000);
}

void start() {
  digitalWrite(9,HIGH);
  delay(1000);
  digitalWrite(9,LOW);
  Serial.println("Starting recording.");
}

void stop() {
  digitalWrite(9,HIGH);
  delay(1000);
  digitalWrite(9,LOW);
  Serial.println("Stopping recording.");
}

void loop() {
  start();
  delay(10000);
  stop();
  while(1);
}