Lessons from Drupalcon 2014

Though I have been increasingly more involved in larger and larger web development projects throughout the last five years of my day job and freelancing, I’m still fairly new to Drupal development. So I was excited to win a very fortunately timed ESIPFed scholarship to my very first Drupalcon, Drupalcon 2014.

I expected that open data and open source culture would go together well, and indeed they did at Drupalcon. Though the greater science community has been glacially slow in coming around to the idea of “open,” the folks I met at Drupalcon who came from science institutions were all friendly and generous with their knowledge in the true spirit of open source.

In addition to meeting other Science on Drupal people and getting a better feel for the greater Drupal community – which I now know is awesome and welcoming – I arrived at Drupalcon with a parallel mission: learn more about how to work with lots of other people to make great websites in Drupal. Now that I’m moving into full-time professional web development, I feel a strong need to learn from and leverage the superior skills and experience of my colleagues.

For the non-developers in my audience: getting from “I need a website” to launching one takes more than just technical skills. The difference between writing code and delivering a working website that meets or exceeds a client’s needs is like the difference between telling people you like bats and researching and implementing a plan to stop the spread of white-nose syndrome between the bat caves of North America.

To me, actually building a website is the smoothest part of the process. In my experience, the bulk of my time and effort goes into understanding what the client and website users want or need, communicating what you or your team can deliver and how you will deliver it, and adjusting your delivery as things inevitably go awry.

I set out at Drupalcon to learn more about tools and processes that can help make these “soft” tasks easier and more effective in a Drupal development setting. Two of the sessions where I picked up many important tips were the Axure prototyping workshop and the Weather.com migration case study.

Why prototype a website?

Prototyping is a crucial communication tool for the design and testing process, especially for larger and more complex websites and features. It can be as simple as boxy drawings on paper or as involved as a full-scale, interactive prototype that is just a few tweaks away from the final product.

In any case, having something for someone to look at, maybe even click through, helps you gather useful feedback and adjust your design accordingly. Lots of early feedback can help you avoid wasting time building things that are wrong for the client. A prototype is one of several tools you can use to help prevent design conversations like this.

Should I prototype in Axure, Drupal, or something else? It depends.

Dani Nordin, who led the Axure prototyping session at Drupalcon, discussed some of the pros and cons of prototyping directly in Drupal. On the one hand, iteration is much slower in Drupal than it is on paper or in a lo-fi wireframe. On the other hand, Drupal, with its highly structured content models and interrelated everything, can be tricky to emulate in a prototype.

So when do you prototype in Drupal, and when do you use something else? First, you have to consider what the prototype needs to be able to do. Nordin pointed out that you don’t have to prototype everything, and some parts of a website are more important to prototype than others. You might focus on prototyping complex or unusual functionality that is difficult to translate into words, unique content, or chunks of development that would be a huge pain to undo if they didn’t work out.

Another prototyping question: who are you prototyping for? Axure, for example, is a sophisticated tool for quickly constructing and sharing interactive prototypes that can give you a pretty good feel for the navigation, content structure and interaction within a site – if you understand what you’re looking at.

Axure Page Layout
Designer’s view of an Axure prototype, including some navigational and interactive elements and reasonable mock content. From axure.com

I’ve been on the client end of a project that used Axure extensively for wireframing and information architecture review. I get the sense that Axure prototypes:

  • Are quick and easy to throw together
  • Generally facilitate communication between development, design and user experience professionals
  • Need a lot of accompanying narrative and potentially some real content to elicit useful feedback from clients

It’s easy to see the professional utility of tools like Axure at the lo-fi wireframe and interaction prototype stage. I am not, at this stage, mucking around with the colors of boxes and the sizes of drop shadows. As an information architect or interaction designer, I’m thinking about what kind of content needs to go where, what it should do and how it should relate to other content in the site. As a developer, this is exactly the kind of information I need to start building a site.

With my user experience hat on, I can further appreciate having the ability to use Axure to prototype and test quite a bit of site navigation and interaction before any part of the site is even built. Early testing can help me avoid a lot of headaches later on, such as when I realize late in the process that I need to totally rearrange the content and break all sorts of relationships and structures that I’ve already developed around.

But you may find it difficult to engage clients with lorem ipsum, empty boxes and Heading 1 placeholders. For people who aren’t design- or development-minded, there just isn’t much in a lo-fi prototype to react to. Much of your most important client feedback may not come your way until your clients have seen real content in your design.

And realistically, real content in a real design may not (probably won’t) happen until after you’ve already made some design decisions and development commitments. As Nordin put it during the session: “Who has the content before they start building? No one? Okay.”

(Aside: In case you are wondering how one designs a website without the content (words and pictures) that go into the web pages: one way is to use some sort of templating system. Content management software like WordPress or Drupal is essentially a sophisticated website templating system that web developers can customize. Someone with zero web development experience can then use these templates to create and publish new pages within a website.

Templates can be as straightforward as nice borders surrounding big blank boxes that people fill in like stationery. More abstracted templates anticipate the structure of future content and include spaces and behaviors for each distinctive piece of content.

Designing for a content management system is kind of like making a packing list for a 3-month backpacking trip. You want to pack as little as possible and re-use as much as you can; but you also want to pack enough to be reasonably comfortable and well-prepared for what lies ahead.)

Finally, if you’re building a Drupal site, you are probably building a site for clients who are going to be managing the content themselves. Drupal content management being what it is, your client may sign off on a prototype that leads to you developing something that turns out to be a nightmare for them to manage. To really get at the implications of certain design decisions, you have to test the Drupal content management interface with the content managers. That you can do only in Drupal itself.

Going big with small victories

I love studying how people solve big problems. Problems that are big in terms of sheer volume, and big in terms of complexity.

So I was fascinated by the technical implications of this truly big problem: migrating the world’s largest website onto Drupal.

The Weather Channel’s website has to serve up more than 2 million different and constantly changing forecasts in the U.S., and lots and lots of bandwidth-hogging media to boot. What’s more, the system has to be able to hold up to huge surges in traffic so it can continue to provide crucial emergency information during major weather events.

The answer to all these challenges wasn’t wholly contained in Drupal. Part of the performance solution that MediaCurrent and weather.com came up with was to chop up page templates into modular content and serve up different pieces simultaneously to speed up delivery. In other words, when you load a page on weather.com, these three things happen:

  • Drupal delivers a page template
  • AngularJS and edge side includes rewrite the page as it goes out to the browser
  • Data services layer delivers additional data to populate content

The page templates are optimized for caching and are cached at several locations around the U.S. by their content delivery network, Akamai, to further speed up performance.

Performance aside, it seems like the most compelling reason TWC went with Drupal was for its powerful and flexible options for content management. Among other things, the team was able to develop an ingenious system that blends widgetized content and custom additions to Panel module functionality. Content managers at TWC can use a grid system to configure re-usable templates and design variants, and it’s all designed to adjust to different devices and maximize re-use of content and templates.

So, I digressed into cool technical challenges. But I also had major takeaways from this presentation that weren’t technical: the importance of relationships and small victories.

First of all, the solutions architect from MediaCurrent said he invested some time in getting to know the in-house web developer team to better under the resources they had available. During the session, the rapport between him and the development lead at TWC was obvious.

Second of all, this team churned out a small side project early on: switching everyone over to content entry with Drupal. The switch led to a drop in support needs and an uptick in publishing velocity that validated their choice of Drupal for their platform – a quick win.

Website migrations, especially huge website migrations, have a lot more baggage and paint points to them than building a similarly-sized website from scratch. The more ground you’re covering with more people, the more can go wrong. I’m sure having an early, bite-size victory under their belts injected some momentum and positive energy into the project that buffered the team against the more grueling tasks that came later and are yet to come.

Getting it right with a project this large and complex takes a lot of close communication and careful planning. In that context, it pays to invest in good relationships and communication with your collaborators on large projects. You’re going to be stuck with those people for a while, and you’re going to be stuck with them even longer if it doesn’t go well because you’re not talking to each other.

As I mature as a developer and graduate to tackle ever-bigger and riskier projects, I’ll keep reminding myself that it’s important to aim for small victories and invest in my working relationships.

WordPress to Drupal 6. Magic! Test post. Now messing with title.

UPDATE 2: Successful test! Manual import showed only one “updated” node, the one corresponding to this blog post. Content did indeed update to match first update. The other 9 TEST Top Stories were “new”. All 10 retained original “Authored on” date. Next test is to change the title of the post from “WordPress to Drupal 6. Magic! Test post.”It shouldn’t create a duplicate as I have set the GUID to be the guid!

UPDATE: Realized that Content > Feeds and Site building > Feed importers correspond to different modules. At this point, duplicates are resulting from the FeedAPI module (Content > Feeds) and the Feeds module (Feed importers) operating simultaneously. Now testing how the “replace existing node” setting in Feeds works. See if this update shows up in TEST Top Story on manual import!

This post should show up as a post of content type TEST Top Story on the dev version of my company’s Drupal 6 site that I’m working with a web support team to troubleshoot. I duplicated the feed importer configuration on the site and set it up to ingest content from my blog and convert each post into a node of type TEST Top Story. I also added a custom image node to my feed like the one that I added to my company’s WordPress blog.

Please note that I changed the parser to look for a GUID to populate the GUID field instead of simply duplicating the link. I’m hoping this will address the content duplication problem we’ve been having.

Content in the existing Top Stories feed and sorts by Authored on date – which corresponds to the day and time of content import. This should not change because we don’t want blog posts suddenly jumping to the top of Top Stories when we update them.

You should be able to manually import content into the TEST Top Story content type by going to Content list > (filter to TEST blog feed type) > TEST blog feed feed and choosing the “Import” tab.

I’ll add a random featured image here for fun.


Don’t worry, I will delete when test is done.

Latin for “testicles”

From a NYTimes interview with Katie Couric, “Katie Couric Has a Few Regrets“:

At your first job at CNN, the head of the network, Reese Schonfeld, famously said you just didn’t possess the gravitas to be in TV news.
Which I think is Latin for “testicles” by the way. But to give this some perspective: I was 23 years old.

For the 15 years you co-hosted “Today,” no one seemed capable of writing about you without using one particular descriptor. Tell me about your current relationship with the word “perky.”
It used to bother me because I thought there was a sexist undertone to that word. It meant shallow and cute, but not somebody who had any depth. It did become a pejorative word, but listen, it’s better than “bitchy.”

-Andrew Goldman

Tsunamis and the energy of atomic bombs

Despite the title, this is not about the shaky state of several nuclear power plants in post-tsunami Japan. Rather, I want to fawn over Kenneth Chang’s piece, The Destructive Power of Water (NYT 12 Mar 2011). An excerpt:

A typical bathtub holds 40 gallons or so of water. That is 330 pounds. A cubic yard of it, filling what at first glance seems a modest volume of 3 feet by 3 feet by 3 feet, weighs nearly 1,700 pounds, as much as the Smart micro car.

And when water is moving at 30 or 40 miles an hour, like the tsunami that inundated northern Japan on Friday, the heaviness of water turns deadly. Imagine 1,700 pounds hitting you at that speed, and each cubic yard of water as another 1,700 pounds bearing down on you. The destructiveness of a tsunami is not just one runaway car, but a fleet of them.

Explanatory science writing at its best. I love, love, love coming up with these kinds of dimensional reference points so that people really get what I’m talking about (Chang, by the way, is an alumnus of the same SciCom program at UCSC that I just finished last year). One of the professors cited in the article estimates the energy of the tsunami was comparable to that of an atomic bomb. Except we’re talking about sheer mechanical destruction instead of harmful radiation. Unless you count the shaky nuclear reactors …

Maybe I shouldn’t be talking about atomic bomb equivalents hitting Japan.

The point is that water is heavy, and it packs a punch. And the idea brings to mind a director describing the filming of an iconic scene in the 1983 movie Flashdance. A huge bucket of water is dropped on Jennifer Beals’ chest while her character, a dancer, is auditioning. Skip to about 1:25 in this Irene Cara video to watch that moment:

You can see her bracing quite hard against the chair, and watch her torso recoil in slow motion at the impact. According to the director, the first time they tried it, they used a lot more water and pretty much crushed her. And it hurt. Like trying to drink from a firehose.

Nuclear War Still Not a Good Idea

A NASA computer simulation shows that nuclear war could throws up a huge soot cloud that warms, rises, and blocks sunlight to cool the Earth. Now that’s the kind of climate change we like to see!

Except there’s also that pesky nuclear winter thing. You know, where the crops die for lack of sunlight and millions of people starve to death.  Not to mention the huge amounts of mutating radiation and radioactive isotopes that cause widespread death and suffering for many years afterward.  What’s more, all that crap in the upper atmosphere helps break down the ozone layer and let in even MORE cancer-causing radiation from the sun.

The original story from National Geographic News gets it mostly right and puts the “nuclear winter” caveat up high. Unfortunately, it has an eye-catching, idiotic headline:

Small Nuclear War Could Reverse Global Warming For Years? (National Geographic news, Feb 22)

Which spawned a ton of stupidly angled follow-ups, some from usually decent news sources:

A Small Nuclear War Would Stall Global Warming (Live Science, 2011 Feb 28)

Reuters picked up the story more than a week later, which is a sign of desperation for copy. Their version of the story doesn’t mention nuclear winter effects till near the end, which is news-speak for “that part is not important or background.” Actually, it IS important, and if you spent five minutes THINKING about the background info, you would realize that putting global warming in the lede and head is a totally cheap scrabble for readership:

NASA: Limited nuclear war could pause global warming (Reuters, 2011 Mar 03)

And it’s just a matter of time until a conservative news source like New American pitches the story as proof that government scientists are crazy:

Govt Scientists Propose Nuclear War to Curb Global Warming (New American, 2011 Mar 03)

Really, this research is about NASA flexing its computer modeling muscles. That nuclear war would be a devastatingly bad thing is decades-old news. The NEWS is that we have a much better ability to describe exactly how bad. Wired, thank goodness, got it right:

How One Nuclear Skirmish Could Wreck the Planet (Wired, 2011 Feb 25)

Yay Wired.com. Boo everyone else.

Apples to apples in interracial marriage

This is a follow-up on a NYTimes story, Black Women See Fewer Black Men at the Altar. The range and extremes in this cursory analysis of interracial marriage rates are pretty striking:

Of all 3.8 million adults who married in 2008, 31 percent of Asians, 26 percent of Hispanic people, 16 percent of blacks and 9 percent of whites married a person whose race or ethnicity was different from their own. Those were all record highs.

Well, since Asians are the smallest ethnic group of the four, just by sheer odds we should be marrying outside our race (‘marrying out’) more often than the other groups. But how many of the interracial marriages are due to preference and how many of them are what we would expect just by the numbers?

If your choice of whom to marry were completely independent of race, the chance that you’ll marry someone from another race would be about like the chance you’ll run into someone from that race on the street. If we take the U.S. Census Bureau data from 2004 (most contemporary survey with all four of the largest races), the single (not married or separated) population over age 15 is:

3.6% Asian
13.3% Hispanic
16.5% black, and
66.5% white.
So since 96 of every 100 single people (13.3+16.5+66.5 = about 96) in the States are not Asian, you’d expect about 96 of every 100 Asians to marry out.

But only 31 of each 100 did, so it looks like Asians show some tendency to marry each other (‘marry in’) more than they marry out. Let’s look at some ratios to see how the same-race preferences compare across the races:

intermarriage rates
race expected / actual = ratio
96.4 / 31 = 3.1
Hispanic 86.7 / 26 = 3.33
black 83.5 / 16 = 5.22
white 34.5 / 9 = 3.83

The higher the ratio, the more likely that race is to ‘marry in’. Asians are, again, the biggest miscegenators, but not by a lot. Blacks, on the other hand, are far more likely than the other racial groups to marry each other based on what we would expect from race-independent marriage. We can speculate on the contributing factors – prejudice, prison, education, age structure, other socioeconomics – but I don’t have any data to support or refute any of them for now.

By the way, in 2004, the percentage of each racial group married without separation:

Asians 61%
Hispanics 50%
blacks 34%
whites 57%.

Asians are almost twice as likely to be married as blacks.

[In a New York Jewish accent] Talk amongst yourselves.

In addition to the referenced NYT story, I pulled March 2004 Census Bureau Community Survey data from these sources:


From those tables I added the numbers of widowed, divorced, and never married to get the following numbers:

3,623,000 single asians >15 both sexes in March 2004
13,294,000 single hispanics >15 both sexes in March 2004
16,499,000 single blacks >15 both sexes in March 2004
66,408,000 single whites > 15 both sexes in March 2004
99,824,000 single people total in March 2004

The percentages of each racial group that are married are taken directly from the linked tables. Yes, I am leaving out Inuit/American Indian, Hawaiians and other Pacific Islanders, as well as mixed-race. They account for, respectively, 0.8%, 0.14%, and 2.3% of the population, too small a percentage for the CB to have useful data on them.