Hackdays - Leveling the Playing Field

This short note was produced in response to a request from Liam Maxwell made at EHI Live 2012, to investigate how hackdays could help procurement. Reposting here as the PDF is locked behind a paywall.

Contributors

• Malcolm Newbury, Guildfoss, Consultant
• Mahendra Mahey, University of Bath, Project Manager of the DevCSI Project
• John Pyle, Independent Procurement Consultant
• Ewan Davis, Woodcote Consulting, Consultant
• Rob Dyke, Technical Director, Tactix4
• Scott Wilson, Service Manager, OSS Watch and Assistant Director, CETIS
• Eckhard Schwarzat, ValueDecision, Consultant

A “hackday” or “modding” (modification) day is an event where a group of people extractedfrom their normal day to day jobs and are brought together typically to deliver solutions touser problems. There is usually an online dialogue between users and developersculminating in a 2-3 day event, in which solutions are developed and sometimes judged. Hackdays are used to develop solutions to problems that may not be solved by the “business as usual” processes. They are relatively cheap to operate, often conducted on a voluntary basis, they are sometimes organised by small business units and SMEs seeing aself interest in participation. As well generating intellectual property (IP) and solutions, hackdays create networks, relationships and creative energy which is unsurpassed by anyother networking event. However, there needs to be a clear route to market for the outputs of hackdays, for their true value to be realised by the community. The public sector procurement process is singled out as having the most to gain from using hackdays, to levelthe playing field and expose innovation.

This short paper describes the hackday process, examines the potential benefits, risks and routes to market and proposes ways for public sector organisations to use hackdays in support of procurement activities.

Cabinet Office is urged to take action to promote the use of hackdays in procurement.

Hackday Examples

Examples of hackdays can be found in most sectors and even within large corporations. For example the first use of Skunkworks or Tiger Teams originated in the aerospace industry. Typically they comprise projects developed by a small structured group of people who research and develop a project from “soup to nuts” primarily for the sake of innovation itself.

Similar ideas include Innovation Games. Individual employees and contractors have made a practice of participating in hackdays inhigher education , clinical research, and healthcare. A quick look at the work of Developer Community Supporting Innovation - DevCSI (Joint Information Systems Committee (JISC)Funded), Open Source Watch and the, Rewired State and NHS Hackday reveals theimmense potential of these events to stimulate innovation with both closed and open source software.

Case Study : Open Data Platform

The NHS Health and Social Care Information Centre (HSCIC) Open Data Platform (ODP) Hack Week is a good example of how public sector organisations can embed hackdays inprocurement.

“The ODP is expected to replace the Secondary Uses Service (SUS) that is currently provided by BT under the Spine contract due to expire in 2013, but which can be extended to 2016. This includes Payment by Results (PbR) service and is used to create Hospital Episode Statistics (HES) datasets.”

The HSCIC ran a “Hack week” inviting “open access and open technology specialists, to demonstrate and develop the architectural principles and pilot the technology which will form our Open Data Platform (ODP). Using real anonymised data, we will explore real information challenges and generate applications which will demonstrate how value can be extracted from the large datasets.”

The ODP proposal disaggregated the more traditional requirements-driven approach to procuring a “solution”. The intention was to replace SUS and related services, a servicewhich is “greater than the sum of its parts”. There are many components to SUS and the HSCIC recognise this. The ODP seeks to leverage contemporary component-orientedtechnology to provide SUS2.

The benefits to the HSCIC were the proving of market and proof of concept/technologystack. The ‘hackweek’ was valuable for the HSCIC as the ODP was potentially a risky procurement in each of these areas. The high level of engagement from the SME /developer community proved the market interest in the ODP. The existing deployment andutilisation of many of the key components of the ODP will greatly de-risk future procurement.The hackweek also proved the integration and interoperability of the disaggregated component approach both in terms of technology and, most importantly, in terms of the work interfaces between organisations.

In terms of procurement the ‘hack week’ expanded the market by broadening the number of potential vendors who could bid for components of the tender. With many of the key SUS/HES components being provided by a single vendor for so long the market was noteducated in the nuances of SUS/PbR. The benefits to the SME community were manifest in opportunity to learn about the SUS environment, the “secret recipes” previously locked into large contracts and obfuscated behind firewalls (soft, technical and legal). SMEs had the opportunity to work closely with the different user groups around the SUS platform and to form alliances with other SMEs to form soft/formal consortia to bid for components of the ODP procurement.

Hackday Conduct

The humble and purely voluntary Hackday should have a strict code of conduct toencourage and enforce transparency, openness and collaboration for it to succeed. Participants need to sign up or agree to a code of conduct which needs to include:

• willingness to use and create Open Source software and/or creative commons materials to neutralise some the ownership and licensing issues which often restricts collaboration
• willingness to openly pre-declare any personal or corporate IP interests inhackday solutions, which could restrict collaborator exploitation
• willingness to indemnify participants against any future legal issues with IP derived from hackdays
• complying with the declared process of the hackday as specified by the organiser
• encouraging good presentation and documentation of the project on a shared wikisite

A suitably adapted version of this code of conduct would allow public sector procurement to use and benefit from hackdays in pre and post procurement phases.

Imagine a hackday called by users at any early stage of a procurement to clarify the requirements/design of a complex project. Imagine a “best and final offer stage' of procurement augmented by an open hackday attended by all SMEs hoping to be in the ecosystem of the winning bid. Imagine a hackday as a condition of breach of contract which allows for the introduction of new suppliers. Imagine a hackday called to establish new forms of interoperability betweenexisting systems, innovative platforms and a new release of a procured system which claimed high degrees of interoperability at the bid stage.

Hackdays can actively ensure the level playing field sought by the Cabinet Office open standards policy paper.

Plugfests and Connectathons

A related form of event is a “plugfest” or “connect-a-thon”. In this type of event the focus is on achieving practical interoperability between systems. These events are typically run withthe voluntary participation of companies that need to test the ability of their systems to work together.

For customers, these events provide a practical means of assessing the level of real interoperability in practice between systems rather than taking supplier claims on face valueor relying on compliance documentation. It also provides insight into the practical value andlevel of maturity of particular open standards.

For suppliers, a plugfest-style approach may obviate the need to engage with expensive certification processes that can exclude SMEs and open source solutions. It also enables suppliers to gain access to data, content and systems - such as those of competitors, or customer"s internal systems - they would not normally have access to for purposes of practical interoperability testing.

Plugfest-style events need to operate under similar rules to other hackdays. For example, CETIS has facilitated a large number of “CodeBash” events in the education sector; although CETIS disseminates the outputs of the events, the specific performance of individual participants is not revealed to those that did not participate. Bug reports and technical issuesare fed back to relevant standards and specifications bodies; a general overview on levels of interoperability achieved is disseminated to the developer community and a list of participants is made freely available to identify those developers and vendors who have demonstrated a commitment to creating interoperability. All participants are free to publishing their own reports on a CETIS CodeBash, however they are strongly discouraged from publicising the performance of other developers and potential competitors.

It can also be beneficial for the process if events are organised and run by a neutral andtrusted entity, to ensure the right parties are invited, to moderate the process and create theright kind of atmosphere.

Integrating the Healthcare Enterprise (IHE) has been running regular connectathons for itsmember firms for over 10 years. Connectathons are highly structured plugfests, moderatedby a suite of open source testing facilities, capable of connecting systems together and exchanging data through one of a predefined set of “profiles" which define the required orchestration of participating systems. The IHE profiles cover a range of healthcare specificactivity, such as scheduled workflow in radiology, pathology results transfer and clinical document sharing and user identity management. The connectathons are open any supplier with products capable of conforming to one of the IHE profiles and the profiles themselves are subject to open governance. Only positive performance is reported and published at theend of the events.

When interoperability is a critical requirement for a solution being procured, our experience leads us to suggest that public sector organisations run these types of event during the procurement or pre-procurement process, and take note where potential suppliers areunwilling to participate. In addition, running these events over the life of an implementation project are important for identifying problems and evaluating progress, and so engaging inthem should be a requirement for suppliers whose systems or content is expected to interoperate with the rest of the solution.

The Benefits of Hackdays

Many of the developers and users at hackdays are actively working to put solutions into projects and products as a result of participation. Success for a participant in a hackday cantake many forms, but getting a solution to the hackday shortlist as well as the recognition and respect this brings from their peers and the personal kudos this creates, can be rewardin itself. However the monetary reward of providing an in-production solution for customers isoften the short to medium term benefit that many seek and the current procurement process does not provide any guidance for the use of hackdays.

Hackdays are an example of re-energising, creative and rapid analytical thought to addressreal user problems in areas where innovation appears to have faltered. They focus onrequirements clarification, design testing, integration and rapid prototyping, can take users,buyers and suppliers to a new level of understanding and create new intellectual property inthe process.

The benefits of hackdays are:

• understanding the role of open source in supporting “collaborative innovation”
• raising the energy levels in both procurers and suppliers to “do something”
• networking for future collaboration activities
• understanding of requirements, issues, and standards for the sector
• understanding of IT skills, standards and platforms
• understanding of what is easy and what is hard to do
• new intellectual property in the form of code and documentation made available for derivative exploitation
• clarity in the attribution of generated IP to the suppliers and procurers who participate

The soft benefits (1-6) should not be underestimated, particularly in a project or organisation,where innovation is severely lacking due to elimination of competition in the delivery phaseof procurement, or during the early stages of procurement when suppliers are afraid tochallenge stated requirements.

Item 7 provides hard evidence of the value of the hackday. Hackday code is intended to befreely shared, downloaded, used or extended as necessary in production systems or products, with attribution (8) given to the parties involved in development.

A hackday code of practice would normally define these terms as conditions of participation.

Using Hackdays in Procurement

The UK public procurement process has remained virtually unaltered for 15 years, whilstincreasing in scope to cover all phases of end-end service design, implementation, deliveryand management in an attempt to ensure a through life cost comparison of bids. But buying services underpinned by software and other digital media has some unique issues:

IP.

Unique opportunity for intellectual property creation exists during in the early stage of most software procurements. Currently, everything in a contract is protected by commercially restrictive terms, but how much of current contracts need to beunique IP? Unique IP is normally only explicitly recognised and valued if it becomesan ‘issue’ in contract termination or novation to another supplier. Rarely is genuinelyunique IP identified and attributed early in procurement, and this represents a huge loss in value creation for UK software Plc. NPfIT is a prime case in point: many of theideas in NPfIT were revolutionary in 2002, but due to being locked away under commercial wraps, its value has since evaporated. Hackdays would have examinedthe requirements in an open forum and identified the unique IP at an early stage.

Standards.

Current use of open standards in procurement to define software behaviour is immature compared with other goods and services, making it difficult toensure a level playing field through specification alone. Consequently, these complex specifications result in “big system” approaches for bespoke systems with no competitive source of subsequent support and maintenance. NPfiT, for example, took far too long in the post contract phase to define its own closed architecturestandards, understood only by a small club of contracted suppliers, causingknowledge and delivery bottlenecks. Hackdays can be used in the pre-contractphase to quickly disaggregate complex requirements into open data, open standardsand open and closed source software components, which can be specified asseparate lots in a procurement and supplied by a larger pool of smaller suppliers.

Interoperability.

The required Interoperability of products in procurement often goes unspecified and results in significant costs and delays in the integration of productsinto their environment post contract. Again using NPfIT as the example - theunplanned integration of central systems into hundreds of local environments wasone of the key reasons for delay and failure. If the contractors had been required to participate in a regular series of independently managed plugfests or connectathons,the number of successful deployments would have been significantly higher and this open form of engagement could have sustained the vibrant market for health IT thatis now having to be rebuilt.

Client-side Software Skills.

Software problems are difficult to track down and fixwithout a client-side understanding of software and a clear chain of command. The end-to-end “one-throat to choke” approach to procurement and been the justification of many public sector organisations to outsource their client-side software skills, leavingthem 100% dependent on suppliers and the minutiae of contracts. This approachwas central to the NPfIT strategy and caused a denuding of local in-house software skills. Hackdays help user organisations to rebuild trust and confidence in software developers to the point where they can think about rebuilding their internal software teams to make them less vulnerable to supply failure.

Buyer Benefits from Hackdays

But how can users and buyer organisations also gain from hackdays? What’s in it for them? Why should they share knowledge and commit resources to these events? Will EU procurement law, particularly around competition and fairness, allow them to use the outputof hackdays?

From a user and buyer perspective, the knowledge gained from a hackday can in itself surpass any knowledge gained from sifting through the marketing, sales materials andcompliance matrices produced from existing procurements. But hackdays could also be an extremely powerful tool to address the risks of a large procurement:

Use knowledge to build own client-side solution

Buyers convert IP into acustomer project to build a solution with own or contracted developers

Use knowledge gained to disaggregate requirements

Buyers use the knowledgegained to disaggregate their needs into a more fine grained set of requirements,tendered for and delivered by one or more suppliers

Use to score supplier bid evaluation

Buyers could evaluate supplier participation in hackdays to provide an “potential to innovate” score for suppliers.

Use to maintain a vibrant market

Buyers could use plugfests and connectathonsto ensure that their system achieves interoperability with other relevant systems tomaintain vibrant market and flow of data through its supply chain.

Risks from Hackdays

Hackdays also pose a number of risks to suppliers and buyers which need to be managed such as:

• The perceived failure of a given hackday project, or product to deliver certainexpectations in a hackday, may cause a supplier and buyer concerns about theproduct moving forward
• Some suppliers may only have off-shore developer resources, and the cost of resourcing hackdays may be more than those with on-shore resources
• Some suppliers may challenge the ownership of hackday generated IP
• Using Hackdays in the procurement process may increase the risks of a supplier challenge under EU law. EU law prevents public sector buyers from making “discriminatory” selection decisions and bans any “negotiation” with suppliers during key phases of a procurement.

To counter these risks, Hackday organisers would have to ensure that

• any derived hackday information to be used in decision making is openly and fairly derived. Currently, any video , software and data recorded is entirely voluntary on behalf of the project. Such reporting constraints mayimpose unwanted burdens on some hackday organisers.
• Hackdays are announced with enough time for preparations to be made
• all hackday IP would either be open source or protected by participant indemnification, so that it can be safely used or suitably converted in any existing build, project or procurement at any time
• participants sign up to forego any threat of legal challenges based on hackday participation

Limitations of Hackdays

Hackday IP is produced as a result of a high energy effort expended over a short time period, and solutions generated should be heavily caveated:

• The more complex the challenge the more likely the Hackday output is in nature afunctional prototype (available developer capacity)
• Due to the time constraints not all the requirements are understood and incorporated (available users / analysts capacity)
• The solution is not production ready (additional release management requirements,policy constraints)

Market benefits from Hackdays

After a hackday, everyone should take a step back to review generated IP in the context of existing and proposed initiatives to achieve full deployment within an organisational setting. There appear to be two potential routes for suppliers to gain from the IP that spills out from a hackday:

Productise

Developer converts IP into an open or closed source licensable supplier owned software product

Use in cloud service

Developer converts IP into a user accessible cloud based service

There is no value in trying to regulate this activity in the procurement process, as this is the stuff of creativity, risk and business. Suffice to say there is ample motivation for developers and suppliers to commit resources to hackdays.

Summary

More concerted central government action is required to exploit the benefits of collaborativeinnovation delivered by hackdays to help level the playing field for open data, openstandards, open source software and the SMEs which support them. As an example of the action necessary, an “open register” of hackday organisers could be procured via the Gcloud initiative to:

• raise awareness of the hackday process for procurement,
• apply more public sector legal rigour around the hackday process and IP generated
• signpost the organisations capable of delivering effective hackdays in differentsectors.

Another example would be for departmental sector sponsors including Cabinet Office toprovide funds to support hackdays where new ground needs to be broken.

Finally, work to produce new guidance for “the use of hackdays in procurement” to address any perceived risks, would ensure broad take up and maxim benefit.