My news/media appetite has been somewhat stunted over the last few years. While I throw jaded scorn at news-radio bite-sized highlights, they are not much less deep than the blog/web/online-newspaper news/opinion material that I do read. I guess news quickies are not beneath me, but what am I missing out by not reading major news magazines? It turns out that the news media is far from dead, and produces some real gems sometimes.
Here is one long essay at Vanity Fair about California’s state finances and an earlier one by the same author about that of Greece. Here is another long essay at the Washington Post about the newly permitted privilege of buying a handgun in D.C.. Here’s another another long essay at the New York Times, about the Air France 447 accident. Each is a deep, literate coverage of the topic, without gross political colours that permeate so much journalism today. Take some time, and enjoy what the “dinosaur” media can still produce.
Compare:
Monty Python’s Life of Brian and Occupy Atlanta
One can’t help but imagine that a videographer could assemble an entertaining clip from this forthcoming event
6. After everyone has read his key ID information, have all attendees form a line.
7. The first person walks down the line having every person check his ID. The second person follows immediately behind the first person and so on.
8. If you are satisfied that the person is who they say they are, and that the key on the printout is theirs, you place another check-mark next to their key on your printout.
9. Once the first person cycles back around to the front of the line he has checked all the other IDs and his ID has been checked by all others
All you need is a fiddle and a caller and you get a geek square dance. Note by the way how the PGP web of trust concept is itself so untrusted at this event. Every attendee is expected to verify every other attendee’s paper ID. They don’t trust each other to validate the IDs. The web of real trust is just one hop deep.
Brendan Gregg, kind enough to engage with us systemtap developers occasionally, posted a long comment about his experiences with our tool. Since his observations appear in good faith, I’m happy to respond to many items in kind. I’ll be brief to avoid tl;dr.
A common refrain is problems with older versions and/or older distributions, which have been fixed for some time. This pattern is not obvious because Brendan’s article lacks version numbers throughout (with handful of exceptions).
Installation. Brendan lists eight steps for installing systemtap on his machines. It is hard to verify all of them since his article does not list version numbers. Ubuntu kernel ddebs may be 610MB, but RHEL5/6/Fedora and clones are more like 200MB. Patching current systemtap releases is unnecessary, and ill-advised if coming from unofficial sources. Upgrading kernels solely for systemtap is unnecessary.
Manual Patching. Brendan’s experiences trying to hack around version-matching safety protections were misguided, considering he realized later that his downloaded debugging data didn’t match his kernel. The build-id error message should be improved to better explain what’s going on.
Execution. I don’t know what caused Brendan’s problem here. Again lacking version numbers or more complete crash information, one might speculate that it might have been one or another systemtap bug, both fixed three releases ago in 2010. Or it may be related to the version-matching safety protection hacking above.
Safe use on production systems. Systemtap has many satisfied users on production systems, so with proper care and testing, it is fine. There have also been problems, sometimes bugs in systemtap, and often in other parts of the system stack that we cannot control, which can make the overall experience less robust than we would like. My impression is that commercial support relationships make it more likely for the whole stack to be improved.
Most of the Time The “ERROR: kernel read fault” message is a soft error in the sense that it represents a caught problem at run time. Sometimes this is a debuginfo quality (gcc version sensitive) issue, sometimes a tapset fragility one. We’ll improve this aspect, for example via this bug .
Screenshot Actually, the “write” and “bytes” warnings are proper, and the rest of the warning message makes it obvious that this was a typo. The intended variable was “write_bytes”. The mishmash of scripts on the wiki are not representative of the regularly tested set that comes with systemtap. Current versions may be seen here.
Uh-oh… It looks like this disktop.stp sample script is in need of an extra few words of clarification. It is “reading/writing disk” from the point of view of userspace. Other scripts in the io group use different metrics.
iostat-scsi.stp The error message for missing guru mode needs to be improved. However, Brendan misread the script as to what fragment actually requires -g. It’s not the %( %) stuff he quoted, which is for conditional processing. It’s for this block only:
%{
#include <linux/blkdev.h>
%}
function get_nr_sectors:long(rq:long) %{ /* pure */
THIS->__retvalue = blk_rq_sectors((const struct request *)(long)THIS->rq);
%}
vfsrlat.stp / WARNING. That error could not have come from the most recent release of the tool.
There’s More… Much of what Brendan was seeing may be explained by old versions or distributor packaging errors.
function(”*”) crashes I agree, it is disappointing that this still is not stable. It’s only a small consolation, but as far as we know, this is not a systemtap but rather a kernel problem. One may reproduce it easily with perf probe. Another way to adduce this is to run a script with -DSTP_ALIBI, which compiles out any probe handler-related code, so as to produce only a skeleton kernel module. If that fails, chances are high that a weakness in the underlying linux kernel (e.g., kprobes) facility is responsible. This has proven to be a tricky area for upstream linux developers to make fully robust. We generally advise against using very broad wildcards like this one, for this reason.
no profile-997 In linux, the standard system timer interrupt’s frequency is not variable. Using the perf-counter related probes, one may approximate odder profiling rates.
non-intuitive In systemtap, one may formulate Brendan’s script in the exact same “two explicit probes” style. The @entry syntax provides an alternative for those who find it more natural / expressive.
Incomplete documentation We suffer from perhaps too many bits of documentation rather than too few. Sometimes new features don’t get added to every place where they’d be appropriate. In this case, @entry is documented in the stapprobes man page, alongside information about general context variables and function-return probes.
Brendan’s conclusion included:
I'm expecting a response from the SystemTap developers will be: "it's better on Red Hat Enterprise Linux".
To the extent this is true, it is because Red Hat takes this problem area more seriously than most other distributors, and goes to a great effort to make the experience as smooth as possible. Neither Red Hat nor the remainder of the systemtap community is in a position to change packaging policies and practices at other distributions. While this may seem foreign to vendors who are used to complete development ownership of a single product, in linux we suffer and benefit from greater diversity of a much greater community.
My family lives in a small town (population 90000), but is near several larger ones that we can visit for cultural stimulation. One regular destination has been the Kitchener/Waterloo The Museum. I fear though that this may have been our last trip there for awhile. The problem? This little beauty, and the audacity to use it.
The story begins with a visit to the museum basement, dodging a birthday party in an adjacent room. (The museum has shed its former name as the “waterloo regional children’s museum”, but wisely not its focus on child-oriented displays.) As a part of the excellent RAM show, we find a completely dark room, with black-light-illuminated circles on the ground, inviting people to step inside. Overhead, speakers belch out some cool special effect noise at the appropriate moments.
Then it happens. A kid asks “how does that work?”.
We try not to panic. We are technologists, and we are in a mini science museum with a bold vision/value statement that includes experiential learning. Let’s do some learning and experiencing. Out comes my pocket flashlight to take a look around the darkness. Aha. A pair of Microsoft Kinect controllers on the ceiling, and a couple of speakers mounted nearby. Aha, there is the totally standard room light switch, right beside the dimly visible exit door. We’re the only ones in the unattended display room, and the booming noises are scaring the brats anyway, so I flick the light on to take a closer look. Aha, there are the wires that connect those bits to something behind a routine shade/stand curtain standing at one side of the room. Do we, or don’t we? You know what I’m asking. Do we draw the curtain back to take a look? Would you?
I am sorry to admit that our curiosity gets the better of me, and yes, we do take a quick look, finding two completely unprotected Macs with some neat screen display of the Kinect data, and some amplifiers, immediately behind the curtain on a table. Out comes some speculation about how it works, comparing screen imagery to the position of people in the display space. The curiosity is satisfied. The room is returned to its previous dark self, curtains restored, machinery undisturbed, and we’re off to see the rest of the show. Later, the incident I just described will be referred to as OFFENCE #1.
Next stop, the top floor, with more RAM displays. The brats are getting a little more antsy, but things are still more or less under control. They are curious about some of the knick-knacks and widgets, and keep their hands mostly to themselves. The displays are again mostly unattended, except by one kind young lady who summarizes one display to us, then walks off. A few of us are taken in by an interactive display with a camera embedded in a screen, which renders an artsie brushstrokey version of whatever the camera sees. Neat.
Then there is a loud crash from behind us, plus a child’s scream. Unfortunately it’s our 4-year-old child screaming. His hand is trapped underneath another display that fell over. “Darn, this may be expensive” is one thought, and “how did he get away from us this time?” is another, and “I hope he’s OK” is a third. As parents and staff converge and lifts the display, Juimiin takes Stuart off to check him out and to check him out and let him mope. He appears only slightly injured. None of us saw exactly what happened – except maybe security cameras whose footage we have not seen. Stuart later said that he only touched it a little bit. I’m not sure I believe him, but he’s mostly OK, so I advise the staff not to worry about him; joke that “he had it coming”. We relax because I see that the display machinery core was not actually damaged: the plainly exposed Mac laptop and the display were still doing their thing.
I help get the display back into its original upright position, and wonder how the heck they expected this one to be safe with kids around. This display consists of a large heavy metal-grill-enclosed LCD screen held up by a 12“x4”ish hollow wooden tower of about four feet in height, with a support structure of another 4“x8”-ish wooden support tower of somewhat shorter height attached to it in a vertical T shape. The two towers are – or were – attached to each other by little more than a few biscuit-reinforced butt joints. The whole thing weighs maybe thirty pounds, and is amazingly top and front-heavy. There is no sign that it was ever anchored to the floor or the wall. Stuart later says that he barely got out of the way. A smaller child might have been killed.
But while we ponder the fortunate misfortune, which we shall later refer to as ACCIDENT #1, a staff lady approaches, and advises that she’s glad everything’s OK, and that there’s another RAM display in the basement we might like to see. We tell her that yes, we have already seen it, we turned on the lights to see how it works, that it’s great. She pauses. Paraphrasing follows. “You turned on the lights?” “Yes, just for a minute.” She looks worried. “Relax, everything’s fine, back to normal.” She heads off. A few minutes later, a staff man approaches, and assumes the most ingratiating and patronizing tone. “So, how are WE doing today? Did WE look at the display in the basement? Did WE turn the lights on? Did WE touch anything?” I’m confused then confounded then annoyed by his first person plural, but try not to show it: “We took a quick glance, that’s all.” “Do you mind if I go downstairs to take a look?” “Go ahead.” Or maybe he said “WE” again. Off he goes.
The tone and the implication that some OFFENCE has been committed downstairs does not sit well with us, especially considering the still-whimpering child. But the brats still want to see some of the other floors, so we head to floor #3, where the science-centery stuff is concentrated. We do not notice that we pick up a tail, in the form of a young staff person.
We walk around, enjoy the usual stuff. Then we find a display that is safe, handsome, tempting, and very slightly non-functional. Pressing the “Start” button doesn’t “Debut” anything.
Eric finds broken things vocally disheartening, so I figure it’s worth a quick look to see what’s up with the machine. I pull out my trusty flashlight again, and look at whatever’s sandwiched between the two halves of the display. Wires, some circuitry visible, nothing obviously wrong, no power switch. I direct a bright white light at the cooling grille, checking out the barely visible internals. Then we move on. Very soon, this event becomes known as OFFENCE #2.
For those keeping score at home, we’re up to OFFENCE #1, ACCIDENT #1, and OFFENCE #2.
A few minutes go by, wherein we make work some driving simulator widget, which the brats enjoy, and show some limited competence.
A few more minutes go by, at which point the “WE” staff guy shows up and starts up a conversation. Paraphrase and immaterial elision follows. “Accidents occur (“ACCIDENT #1”), we understand that. But security camera footage shows that you looked behind the curtain downstairs. (“OFFENCE #1”) And now you were seen attempting to fix that display over there. (“OFFENCE #2”). It is not your place to do that. If you can’t stop yourself from meddling with things, perhaps this place is not for you. My CEO advises that WE can refund your admission, and WE can show you to the exit.”
Those of you who know me will not be surprised that a bit of verbal combat is not intimidating. “Regarding OFFENCE #1, what exactly is the problem? Did we damage anything?” “No.” “Is everything back to normal?” “Yes.” “Was there being anyone else there, being disturbed?” “No. But the artist’s vision ….” “Was any harm done by our curiosity?” “No, er ….”
“Regarding OFFENCE #2, what exactly did I do?” “You used your flashlight, as if you were about to try to fix it.” “You mean I only used the flashlight?” “Yes.” “And I didn’t actually touch anything on the inside?” “Correct.” “So we did nothing but bombard the hardware with some extra photons out of curiosity?” “Yes.” “Do you have something against curiosity?” “We support curiosity, but …” “But not too much curiosity apparently. What a pity.”
“Have a good day, Sir.”, may have said Mr. WE, and walked off, leaving us alone. Juimiin and I stared at each other in stunned silence. This was not to last, as the brats didn’t understand what was going on, and just wanted to get back to the game. I overruled the first instinct to just leave the darned place, so instead we pretended all was normal. The brats played at one or two stations for maybe fifteen minutes more, but we were keenly aware of the “tail” still coming and going, keeping an eye on us. I wonder if we were one wrong flashlight-activation away from being dragged out by burlier men. We did leave a little phosphorescent graffiti in their light-sensitive “shadow image” room as a goodbye present from the flashlight. It just said “Bye!”, which the following image (from another venue, earlier) only vaguely portrays:
The morale? Darned if I know if this is the message the museum meant to send, but this is what I received: Don’t be too curious. Don’t let your kids accidentally break stuff (which by the way we’d be willing to pay for repairs to, if called upon). For that matter, maybe your kids would be safer at home. Most of all, leave your flashlight in your other pants.
UPDATE: David Marskell responds.
As a response to my last story, I received an email from the “THEMUSEUM” CEO, David Marskell. To his credit, it was courteous and detailed.
I will not post his email without his permission. However he made some points that are worth commenting further upon.
To Mr. Marskell, it appears the root of the problem was a “disconnect” between our reference to the facility as a “mini science centre” and what he views as reality. He compared the RAM display to art galleries such as the “AGO in Toronto or the Louvre in Paris”, insofar as the sanctity of the artwork is concerned – and perhaps also their putative value. To the extent this may be true, the problem originates in the half-baked rebranding of this facility.
Until recently, “THEMUSEUM” was known as the Waterloo Regional Children’s Museum, and sported 5.0 floors of kid-friendly displays, including travelling exhibits. Now, it sports 3.5 floors of the same kid-friendly displays, plus 1.5 floors of travelling exhibits. Previously, it offered cheap entrance to entire families, with deep discounts for children, including to the special exhibits. Now, it does exactly the same. Previously, the place was full of kids, including toddlers. Same now.
I get the impression that the organization is trying to have it both ways: retain its family clientele, and yet bring in the adult art aficionado. The problem is that if one squishes all that into a single facility, incidents such as ours will inevitably happen. Putting fragile & dangerous & high-putative-value widgets within the unsupervised reach of the general family-weighted public seems delinquent in foresight and due care. (When the Titanic artefacts visited, many were properly enclosed & alarmed, so they get this right sometimes.) Berating people for being curious about how something works — even if it’s “art” — seems like a polar opposite of the mission and history of the place.
Over time, I expect “THEMUSEUM” will untangle its mixed messages and fix the undisputed safety/security problems. Maybe they will become a little less resentful toward those who accidentally highlight how far they have yet to go.
There are many noteworthy bits of art. There are many silly ones. There are not a few that attempt to be so avant-garde that they are unintentionally pathetic.
So is the case of one Marni Kotak who decided to give birth inside an art gallery. The article offers that this was the artist wife’s and artist husband’s first child. This means that they have managed to perform the defining natural function of life, reproduction, for the first time ever! Congratulations!
But on the other hand, there seems to be a lot of reproduction going on all around. What, a great many humans are born every day. Often, the family is or can be present at the event of this minor miracle. Human births must have been witnessed about as many times as pianists have rehearsed scales for warm-up. One might opine that births are … rather common, actually.
That is, in the normal parts of society, the parts that create life and build families. There, reproductive biology is not some sort of abstract platonic archetype that we may only hope to reach by paying to see “artists” doing things naked. It just is. Never mind what kind of person would put herself on display; what kind of people would pay to see it as “art”? Maybe they just live smaller, simpler lives than the rest of us.