Marginalia 7

Intelligent artifice

18 January 2020

Recently I've read and watched some interesting stuff about technology adoption and real life workflows, which luckily fits in very well with my new paid work. Build what you need using what you have to collaborate and empower and change explores what can happen when the outcome of a tech project is prioritised ahead of the technology: and also what can happen if the metrics are prioritised above everything else. It reminded me of Matthew MacDonald's piece Microsoft Access: the database software that won’t die. MacDonald points out that regardless of what professional software developers think of Access, for the average "power user" (a largely ignored group in recent times) it is incredibly useful and works pretty well most of the time. Indeed these articles taken together are a good primer for anyone wanting to move a large group of people from using one technology to another. Why are we still using MaRC in libraries, for example? There has been no shortage of proposals to move to something else - whether some kind of Linked Data, or at a bare minimum storing MaRC field data in XML or JSON formats by default rather than persisting with the esoteric and outdated ISO 2709. There are many answers to this, but a large part of the problem is simply that MaRC still does the job adequately for the the small, suburban public libraries that make up the vast majority of the world's libraries. Sure, it's far from perfect, but it's also not so completely unusable that there is enough incentive to move away from it, especially with several generations of librarians having learned to read, use and manipulate metadata stored as MaRC, and a (albeit shrinking) marketplace of library software that is built around MaRC records.

We can also see the reverse of this when looking at institutional repository software. Libraries in Australia and elsewhere appear to be moving out of an initial exploratory phase of diverse solutions using mostly open-source software, to a consolidation phase with the institutional repository 'market' shrinking to a very small number of proprietary systems. There are many reasons for this, and I'm by no means an expert on the ins and outs of repository software, but it seems this is primarily a response to the costs of DIY system maintenance and troubleshooting. Whilst open source advocates often speak about the freedom provided by FLOSS, we all know that with freedom comes responsibility. In the case of Australia, a very diverse range of software and version of software has meant that the mythical "open source user community", let alone commercial support, simply hasn't existed in sufficient numbers for any given system: especially when we're in the "wrong" timezone compared to most potential overseas supporters. This fuels the perception that using open source means needing full time tech support on staff - which is sometimes possible but often tenuous in the higher education funding environment. I've spoken about this before in the context of local government - there's a reason managers often ask "who else is using it?"

But here's the thing: just because someone else is doing the maintenance and writing all the code doesn't magically make the software more sophisticated and reliable. Most software is chewing-gum-and-string when you peek behind the curtain, and it's the boring old stuff that is the most robust. This is the case for all technologies: trains kept exploding, derailing, and colliding for decades before adequate safety systems were invented and rolled out. The first commercial passenger jet, the de Havilland Comet regularly disintegrated mid-air. Microfiche is arguably the most reliable dense storage technology for text and two-dimensional images.

In fact, the sophisticated "Artificial Intelligence" systems that venture capitalists, tech journalists and incompetent public service chiefs are falling over themselves about are often no more than a vast network of humans behind a slick website.

Ben Tarnoff talks about what actually happens when human interactions are automated in Discipline at a distance:

Creating a hybrid of the factory and the putting-out system is feasible because networked digital technologies enable employers to project their authority farther than before. They enable discipline at a distance. The elastic factory, we could call it: the labor regime of Manchester, stretched out by fiber optic cable until it covers the whole world.

Tarnoff is primarily (though not exclusively) talking about services like Uber or Deliveroo, where workers have 'flexibility' as 'independent contractors' but are subject to the discipline of and receive much the same pay - and to an extent, conditions - as a nineteenth-century factory worker.

But to return to the issue of technology adoption in less brutal employment conditions, the most important consideration is not really the inherent 'sophistication' of a technology but rather how easy it is for a given organisation to maintain it, given the resources the organisation is willing and able to access.

The trick here is to be able to identify the point at which it is better to invest in enabling staff to learn a new tool vs continuing to use "what you have". If you can increase "what you have" (i.e. by expanding what staff know) then options about what you can use become broader. This is essentially what my current job is mostly about: Putting the ‘tech’ back into technical services and the rest of the library.

"What you have" is not only contextual to individuals or organisations but also reasonable expectations vary by industry and profession. This is one reason why so many workflows designed by programmers use git - it's become an industry-standard tool allowing a reasonable expectation that most if not all professional software developers will know how to use it. But this runs into reality once we move into other domains, particularly where there is cross-over: for example documentation workflows. git is a powerful tool for distributed version control of written texts: but it's also notoriously difficult to learn, can be frustrating and stressful to use, and has such torturous syntax and poor documentation that it's difficult to tell the difference between the parody docs and the real ones.

Dan Cohen notes the problem universities have with being able to match the huge salaries available to talented technologists in the private sector in The nature and locus of research. "By drawing researchers fully out of the academy, we lose not only teachers and mentors, but a style of thinking and research that is different in important ways", notes Cohen. I'm not sure what Cohen would think of it, but I personally think Kevin Sanders' work implementing a .onion site for the University of West London institutional repository is a good example of the 'style of thinking' that is possible in a university, but not in a gigantic surveillance and advertising company.

It also totally rules.