Wednesday, July 27, 2011

A Day In the Life, bSol 2669 (and a Crumpler Countdown [-up])

The days that I'm the TAP/SIE on tactical duty, I’m usually the first one in to the MER-B sequencing room. It’s partly because I might be carpooling with my roommate, who likes to leave before 7 in the morning, though I usually avoid carpooling on these days because of the 50-50 potential of the day going long. Really, it's mostly because I’m a morning person anyways, which scares people. 

The first thing I do when I walk into the sequencing room is turn on the lights. By default they come on at full tilt, and I hate it. I use the dimmer controls to bring them down to something bearable — 10 or 20 percent of normal. (Really.) I’m one of those people who hates bright lights, one of those people who has to stare at a computer screen most of the day, one of those people that downloads F.lux. I’m more at peace with the lights on low.

This is all the result of my early and consistent use of a computer in my life. The people who didn’t grow up with computers — you know, those old timers — tend to like to read things on paper. (One of my friend’s all-time biggest pet peeves: “When someone who doesn’t understand the Internet asks me to print out a web page for them instead of linking it to them. You know, like a smart person would.”) For this whole "reading" thing, they need more light. For this, I reluctantly allow them to beam up the lights in the back of the room, leaving my corner — the TAP/SIE's corner — just dim enough. 

Thank the gods for multiple light sets.

Ah yes, the TAP/SIE corner. Farthest from the entrance to the room, it’s place from where small-voiced requests emanate throughout the course of the day: “RPs, I need you to redeliver your sequences. You broke my stuff.”; “TDL, what’s the magic sol?”; “Where is the coffee intern?”; etc.

I think it’s best this way, with the TAP/SIE all by his (her) lonesome, the RPs dominating the other side of the room, and the TUL and Mission Manager piloting the room from the center conference table. The TDL is the floater: they run back and forth between the other engineers’ offices, their own computer terminal in the corner adjacent to mine, and the center table. Everybody gets their space. Everybody is happy.

Today was just the same. Door open, lights on, butt in chair. I sit down in my corner and set up shop. Laptop -- on and cranking. Phone -- headphones in with some Derek and the Dominoes to start my day. Notebooks -- out and ready for action. Go time.

Wait, hold your horses — an email from the Mars Express mission planning folks just came in. I’ve got to generate some sequence products for them, so it’s off to the races for 20 minutes. 

I come jogging (ahem… figuratively — I sprained my ankle pretty badly a week ago and I’m on crutches, which gets me lots of sympathy and rides to and from the parking lot) back in to MER world, still the only one in the room. Somewhat to my surprise, today is a slide sol. Our relayed downlink from the Mars Odyssey spacecraft has a bit more latency on it than we're used to for "nominal" planning days. Remember, we don’t like to start our day until we know something about the state of the rover. Odyssey is collecting its own science data, too, which the Odyssey team is always kind to remind us — they’re not just there to relay Oppy’s data. The advantage is that we’re starting an hour later than the usual 8:30am, which means more sleep for the humans. The disadvantage is mostly felt by the remote science folks in the Central and Eastern time zones, where they get out by late dinner time on a normal planning day as is. 

Kevin, a recently-certified Senior Rover Planner (he says, “I can now drive without adult supervision”), comes strolling in to the sequence room. He’s not on tactical today, but he is the MOB/IDD downlink analysis goony for the day. “How much data are we getting back?” he asks. I have no idea; I wasn’t on tactical the day before and didn’t have the number in my head. I’m more peeved that he made me take off my headphones, as “The Boy Who Wouldn’t Hoe His Corn” is playing and I hate having that song interrupted. Hate

Noting my hint — a shrug of ignorance — he instead looks at a plot that’s been placed on the center table, and it tells him the answer: 30 megabits in the first chunk, 20 or so after that. The first 30 will contain most of our critical engineering health and safety data, as well as the high-priority images in the assumed drive direction (southeast-ish). Good enough for him. The downlink is coming in right as he scuttles out of the room to go… do whatever it is that MOB/IDD does. 

Soon, Paolo comes in with his "I'm RP today" stride. With him is Albert, today’s SOWG Chair. It's rare to have the SOWG chair in the room. When we do have a SOWG chair right next to us — in a chair, of all places — it keeps us on our toes. He also wore the SOWG Chair hat for the two previous planning days, so he's in the loop. He knows what's up. He's got ideas. Oh, the ideas.

Yestersol’s drive (which, actually, happened only a few hours ago, since it’s 16:50:00 local solar time for Oppy as I type this) landed us on one of the last bands of outcrop before landfall at Endeavour Crater. “A 120-meter approach drive,” Albert laughs. Usually when approaching a band of outcrop or any other target for IDD work, it takes 2 or 3 drives — therefore 2 to 3 sols or 2 to 3 planning days — to land us right where we want. We got lucky today and ended right on top of the outcrop without the need to do additional bump and approach drives. Excellent.

The Navcams and Hazcams have been processed by now (9:15am). Albert was right: that Front Hazcam shows us the outcrop right under our feet.

I enhanced the contrast on this puppy to make Endeavour "pop" out. Oh, by the way, our updated Crumpler Countdown is:

0.50km, 97.8%

125 meters of driving yestersol. We can almost taste Endeavor. 

Our TULs — one prime, one shadow — come in. “Sounds like an IDD day,” they say. They know. They get it, too. Matt Golombek, today’s LTP, comes in next: “Holy cow. A 120-meter approach drive?! Precision if I ever saw it.” The irony of the word "precision" is not lost on him.

The day starts with the Engineering Tagup. I'm not required to be at this meeting, but I keep my ears perked. Meanwhile, I set up my work environment: 14 Unix terminals, 1 Mozilla window, 1 Firefox window (I know), 4 standalone applications, and not enough coffee. It takes about 10 minutes to get it all in lockstep, to get the crank ready for turning after the upcoming SOWG meeting.

The half hour to 10am zips by. It's time for the SOWG meeting. A few PowerPoint charts later, we get Albert's summary: “The three words I’m taking from Matt’s LTP report are: ‘MIs, go wild.’” Meaning, it's a day for placing the IDD on the outcrop and using the Microscopic Imager. According to our database, it's been precisely 58 sols since we last used the MI in this same way (if at all) — sol 2611.

What comes out of the plan is simple, in a sense:

1) MI stack and stereo imaging
2) MB "touches" in between
3) Place the APXS on that damned rock and collect data for as long as we can
4) Get some Pancam images

The devil's in the details, though. And everywhere else. We had to tinker with the activities, the deep sleep duration, and some of the Pancam observations to get the power situation under control. We nuked a Pancam activity to survive the night — we tried a "wake up at 8am LST and do some stuff" kind of thing, and our modeling tools told us, "No." 

Hey, look, a Pancam has come down! The Pancam images usually have lower priority than the Navcams, but they give us range data for the terrain up to about 100 meters. They are what make long blind drives possible. For today, we don't need them... but tomorrow, they'll be handy.

The day passes rather uneventfully. In this sequence, the RPs are shooting for some five different targets on the ground — Gibralter 1-4 and an extra test spot — with some stuff in between. Enough tool changes to make you forget the right-hand rule and everything you learned about coordinate systems in under 5 minutes. During the walkthrough, the animation follows the RP's cursor in the command-building utility (RoSE), and we get all glossy-eyed.

Despite the large number of "backbone" and "helper" sequences coming from the RPs, and despite some other curve balls thrown in by the TDL, I expect sequence integration to go poorly. This is one of the most tenuous parts of my job, where I stitch together everybody's individual sequences into a single integrated sequence for uplink to Oppy. A few command-line Perl and C-shell utilities do most of the heavy lifting for me, but they spit out a lot of messages. You have to know the precise meaning of them, what is ok to ignore, what is ok to scratch your head over and drag the TUL into (number one rule in ops: if you don't know, ask), and what is definitely not ok to ignore. Of all the longest tactical planning days of my life, 90% or so of them were long because of sequence integration. Too many of the secrets and myths and project lore have been lost in the automation scripts that we've written. We often have to go back into our uplink reports database and search for arcane command-line warning phrases or keywords to see what the tactical team did the last time some error came up.

But today, it all gets stitched together without issue. Our sequence modeling and flight rule checking tool, SEQGEN (developed at JPL), seems to like what we've fed her.

This has been way too easy. Something's wrong.

We get to our next meeting, Sequence Report Walkthrough. We're nearing the end of it — this is going way too swimmingly for an IDD day — when, suddenly, Albert says, "Can you back up to the second MI stack?"


"I see a [error]," he says.


"Oh, crap," says our RP team, "We screwed that one up."

A simple fat-finger and some room-wide dyslexia just threw a screwdriver in our day.

"Ok, redeliver," say our TULs.

At this stage, "redelivery" means I have to undo all the stitch and modeling work I've already done, and then do it all again. But, hey, this two-fold lesson is one we learn a few times a week:

1) This is why we have sequence walkthrough meetings.
2) Life is tough.

The redelivery goes well. We see the fix. We're happy with the fix. We move on. Life is well. Everybody goes (limps, for me) home.


Today's wrench-in-the-gears was but a hiccup. In light of the 120-meter pseudo-approach drive, and in light of the beautiful MI images we'll get on the ground in a few days, and in light of everything else going well, it's not even a blip on the teams radar. They've adapted to these things. I, on the other hand, am still getting used to these "hiccups" after even a full year of doing this job. I get this sense of panic, a touch of shame, a bit of consternation when something — however small — goes wrong. Sometimes you catch the team on the wrong day, when our collective tolerance level is low. We get frustrated and we continue to make mistakes.

Today, though, was not one of those days. Smooth.

No comments: