One repository to rule them all and in the darkness bind them

One of the most commonly asked questions these days is “Should all our source code be in one repository?” This is a complex question and leads to a somewhat interesting set of answers.

Before we get to that lets try and understand the question a little more and find out why customers asking this? In IT we like to centralize and optimize. Gathering all the code in one place is seen as the next logical set of distributed data ripe for centralization and optimization. All in one place means we can manage access better, manage backup and recovery better and ensure everyone is able to maximize the reuse of code.

However this flies in the face of modern developer behavior. At large and small IT organizations we see developers downloading open source source-code management systems for themselves and their teams. Instead of having one repository in one place we are seeing repositories on every server and developer hard drive creating a vast digital archipelago of repositories where processes and standards evolve on a team by team basis mimicking the finches on Galapagos recorded by Darwin.

And this is the dilemma. Corporate responsibility drives towards a single repository strategy but developer behavior wants local control and ownership of the their code.

What does corporate want?

So what does corporate really want when they say they want a single repository? Typically they are trying to address multiple concerns and typically these are they:

  • Visibility into all the artifacts in the repository
  • Central access control over the artifacts
  • Conformance to governance guidelines and audit reporting requirements
  • Segmentation of the artifacts to match separation of duties mandates
  • Support for shared code and refactoring initiatives
  • Enterprise wide impact analysis
  • Control over misuse, misappropriation and malicious activities
  • Consistent backup of the repository

None of these are architectural in nature: they are all functional requirements that are easy to satisfy with a single repository and very difficult, impossible in some cases, to achieve with team-based repositories.

What do developers want?

Developers want the least amount of technology and process in order for them to develop at speed. To, as Mark Zuckerberg described it, “move fast and break things.” This means:

  • Solutions they can obtain without budgetary permission
  • A repository that is easy to use and flexible to their needs
  • Low process, governance and control
  • Easy (or no) administration
  • Simple (or no) licensing
  • Fast checkout and checkins (especially GetLatestVersion) across the LAN and WAN

Once again, these requirements are not architectural. They too are just a list of requirements. While they seem in conflict with what corporate governance demands there is common ground and a proper technical solution that meets both sets of requirements is possible.

Missing pieces

Developers fear having their code hosted on a platform that they are not developing for. Mainframe developers would never countenance their COBOL code hosted on Windows, no Unix developer would accept their code hosted their either. Developers in Beijing find it hard to accept their code hosted in Bulgaria and managed from Boston. Add to this the numerous code pages and, perhaps, ASCII to EBCDIC conversion issues that would ensue.

Most developers these days use code analysis tools designed for the development platform they are using so this means keeping the code on that platform and that in turn means duplicating the code from the single repository back to the distributed platforms.

As I said at the beginning this question raises many interesting issues. None is more pressing than this though.

Neither of these positions, single repository versus multiple distributed repositories, takes into account is that the source code repository represents the collected intellectual property of the corporation. It is a business’ most valuable asset, far beyond the goods and services they provide, and this is why it has become the single target and focus of hostile foreign governments, unscrupulous competitors, disgruntled employees and organized crime.

Secure SDLC: the next standard in repositories.

In tomorrow’s repository the design needs to represent best practices in secure data management. Protection of the repository is of utmost importance. This means that our repository must have:

  • Single point of access control
  • Robust auditing
  • Encryption of artifacts
  • Tamper detection of artifacts, logs, audit trails, reports and the software itself

The ideal repository architecture

What makes the ideal repository architecture is neither single nor multiple repositories.

Here are the key ingredients and, as you will see, they satisfy all the corporate and all the developer needs:

  • Secure repository defended against exfiltration and infiltration of code
  • Process centric allowing enforcement of one (or many) development processes irrespective of platform and in support of all development methodologies
  • Secure, immutable logging and audit trails
  • Single point of user and tool administration
  • Artifacts stored on the platform of choice by developers
  • Artifacts backed up by native utilities optimized for that platform
  • High speed performance over LAN and WAN
  • High speed performance irrespective of the user load, irrespective of the size of the repository and irrespective volume of versions and changes being managed and tracked
  • Caching of a minimal amount of code as needed reducing duplication and limiting misuse

We call this a Single Virtual Repository.

From a management and administration point of view it appears as a single repository but, behind the scenes, the SCCM software manages all the artifacts in their respective locations on their respective platforms.

From a developer’s point of view their code is collocated with the team allowing for the fastest possible access. It also means that code analysis tools are able to execute on the code natively without duplicating the code. Each team can have their own, or a mandated process, as processes and access rules can be defined at a project or even an artifact level.

Central control but distributed data.

Posted in Business and Technology | Leave a comment

NYSE, WSJ, UA outages: aren’t we missing the point here

It’s war

When the triple crown of cyber outages in financial services, communications and transportation, occurred last week we all waited tremulously for the next one just as one does for an aftershock of an earthquake.

Today we are seeing the revelations emerging that suggest “glitch” over “hack” in each case but either way the reaction seems all too subdued and way too calm. One report in from HMG Strategy LLC suggests that having a team trained and on standby to remediate the incident’s consequences is all that we need.

Absolutely not! We are all under attack right now. Hostile foreign governments, organized crime and terrorists are recruiting the best and brightest as part of their respective cyber-armies to plunder the cyber-fabric of our society.

We cannot stand idly by accepting the glancing blows of each attack that fails because the one that gets through will knock us out for the count. It’s been calculated by the Ponemon Institute that the average outage in a large, highly regulated organization, costs $500,000.

We must be proactive and defend ourselves against any and every ever threat and we must adapt and anticipate threats yet to be conceived and executed.

Further, we must take the offensive. If a threat penetrates our defenses we must track it, log it, record it with the hope of tracing it to its source so that we can use diplomatic, legal or military force to neutralize the threat.

Make no mistake here: we are at war. In the last three centuries of conflict we have shifted from military casualties to civilian ones, but today the casualty is commerce.

Posted in Business and Technology | Leave a comment

Ordinateurs sans frontières

Lines on paper pre-1914

We are 100 years from the War To End All Wars and the ripples of this global tragedy still echo down through the century into our lives today. The boundaries drawn in 1919 as arbitrary lines on paper shaped all too many of the conflicts of the past one hundred years and are the root cause of many still in the world today.

Paul Mason, of Channel 4, recently asked the loaded question “How did the First World War actually end?” (Thanks to Jonathan Sugarman @WhistleIRL for finding this article). His view, and mine now too, is that it hasn’t ended. The guns stopped firing in Flanders, at Ypres and on the Somme at 11:00 am on the 11th day of the 11th month of 1918. It took just 26 days for hostilities to break out again, on December 7th 1918, in a squabble about the remains of the Ottoman Empire. Since then there have been 285 wars and only 26 days of peace.

As I was cycling to work this morning, listening to the excellent BBC series on WW1, I was thinking about the technology 100 years ago and tried to extrapolate what it might be 100 years from now. I had only fantasies. What about 20 years from now. Still only fantasies. How about just around the corner in the next decade?

Then it struck me that the peace trumpeted in 1918 might finally come soon from technology.

We are a planet of paper borders drawn decades, centuries, even millennia ago, but we live in an age of technology that flies across borders without passports or customs checks. Technology is eliminating meaningful borders despite lame attempts by national and local governments to enforce them. Being able to shop for the best price anywhere on the planet is empowering of shoppers and dis-empowering of corporations and governments. So why do we need borders any more?

Tribes (friends, family, colleagues, alumni, hobbyists, peers, factions) get together online more often than they ever did in person before the internet. Geography and (what we now would call) primitive communications constrained how we were able to participate in the governance of our lives. Representative government was created because we couldn’t all go to Washington but now we can (and do) comment on the issues of the day and participate and effect global changes.

Borders are becoming less and less relevant and the governments that enforce them more and more marginalized. Instead of fighting a losing battle against technology why not embrace it and exploit it.

So this is my prediction: Technology will continue to confound and frustrate governments who will bring increasingly draconian measures to bear on the use of technology across their borders. Technology will respond with innovative ways of operating that bypass and circumvent the restrictions. A breaking point will occur driven by over-regulation, overlapping-regulation and contradictory-regulation until  “de-regulation” will once again be on every tongue. Then there will be a free-for-all and much to exploit until we settle into a new era of commonsense, self-regulating use of technology across borders. Some borders will crumble (North Korea, China, USA’s isolation of Cuba) bringing chaos (and maybe tragedy) to them until they find a new equilibrium on a new world of information and access. Some new borders will emerge organically and transitionally (Catalunya, Kurdistan, one, two or even three new Ireland’s perhaps) based on affinity rather than geography. Cross-border, border-less communities will emerge and operate as quasi-states (Global-Jewish-State, Global-Islamic-State) and will be more representative of their communities as a whole than the narrowly focused, border-bound national governments they are connected to (Israel is not representative of Jewishness neither is Syria/Iran/Afghanistan representative of Islam).

My hope is that technology will empower individuals to define and act upon their affinities irrespective of the ancient and arbitrary lines drawn on paper. Technology enables border-less super-groups to emerge that are more successful in effecting change and combating injustice. Power in the hands of a few oligarchs, patriarchs or autocrats is diminished through the illumination that comes from technology’s light.

I’ve never yet seen a technology prediction that hasn’t been scoffed at one day only to find it trending the next. So it is with some trepidation that I stick my neck out here to suggest a future that flies in the face of human nature to cling to power at all costs. In many ways what I am suggesting is technology that is already here and already part of everyone’s daily life and the shift I describe is well underway.

But there is a major battle, perhaps even a war, coming between technology-openness  and controlling-power. We have seen the first skirmishes (Assange, Snowden). We have seen the first battles (Net Neutrality, Chinese censorship). The forces have not yet engaged all their might: they will.

So I’ll make one more prediction: For the first time in the history of warfare the victor will be chosen by the victims and not the protagonists. Victories will be measured by lines erased from the map. Casualties will be laws restricting free interaction and access and those people and corporations who wield power for their own sake and not “… of the people, by the people, for the people.”

Posted in Business and Technology, Personal experiences, Personal growth | Tagged , , , , , , | Leave a comment

TCP/IP is 40 years old

In honor of the critical technology that underpins everything, not least of which is this little blog, there’s a TCP/IP party on Saturday.

Tickets are $25 and can be obtained here.

There is a great list of speakers:

Vint Cerf – with Bob Kahn, wrote the TCP spec, published in the IEEE Journal May 1974
Yogen Dalal – co-authored the first TCP Specification (RFC 675, INWG 72)
Judy Estrin –  worked on the initial TCP project at Stanford
Charles Goldfarb – Markup Languages
Dan Lynch – Founder of Interop, worked on TCP at SRI with BBN
Nancy Shepherd – Mayor of the City of Palo Alto
John Shoch – PARC Universal Protocol, PUP an important predecessor to TCP
Marc Weber – Internet Historian, Computer History Museum
Sean Askay – Google Earth Outreach

The team will be launching the “Conceiving the Internet” that is collecting stories about how the Internet began.

Posted in Business and Technology | Tagged , , , | Leave a comment

Internet of Everything: Part 3: Data is the new oil

Internet of Everything
Device Data Decision

Edward Snowden would have us believe that everything is online already so worrying about how much personal information is out there maybe a quixotic fear at best.

And it is true, since Snowden’s and Assange’s revelations, we now realize just how much data is out there that comes from the telemetry of our daily lives. And we are suspecting there is much more than anyone cares to say.

Our phone tells the world so much about us. This strikes fear some people who think the government is planning to subjugate them by force. If I were planning to do something very bad, I too would be fearful of the reach of the state into what I am doing. But I am not planning to do something bad, and so I am not worried if the NSA knows I went to the gas station on Saturday and that I called my daughter in Florida who was also at the gas station at the time.

In my view everything, the IoE, cannot be online soon enough! I have so many questions and I want the answers now! Big Data means we can dream big.

Storage is cheap
The coolest thing about the IT industry is the pace of change driving down the cost of technology. Thank goodness the barrier to entry is low in high-tech. Imagine how much progress we might have made if Tech behaved like Pharma and protected its secrets for decades at prices that bear no relationship to the cost of production.

With the cost of online storage fast approaching zero (and for many consumers with cloud-based storage the cost is already zero) the price of storing everything is insignificant. Think of the complexity and difficulty of piecing together one’s family tree back more than a handful of generations. Compare that to our grand-children’s ability to reconstruct our histories down to the daily level as they read through the Facebook, Twitter and LinkedIn archives 20 years from now.

All technology that is conceived today must make it a foundational principle that it reports its telemetry to the cloud. Who knows what insights we’ll get from our internet-connected refrigerator: a correlation between our ice-cream consumption and relationship status perhaps or a warning that the package of burger patties in the freezer has been recalled for containing donkey-meat? But not having that telemetry means not being able to get these insights.

The trend for wearable devices has been a giant consumer success. With these devices connected and delivering to the cloud we have yet more insight into who we really are.

Soon every wearable device will be part of the IoE: hearing aids that stream the TV and Radio sound and translate foreign languages, glasses that see and hear and record and upload (already here), pens that record what is written (already here) and pedometers that remind us to get up and stretch out legs (already here).

It’s all about the questions we ask
The new hot job title is going to be CQO, Chief Questioning Officer. This person will be responsible thinking of the right questions to ask and for creating the technology to answer them.

Every business, great and small, will be more successful if it delivers better goods and services with greater margins than its competitors. The IoE makes this possible. With all that data out there and with everything connected to every other thing creating more data new truths are awaiting discovery.

The IoE matters because it collects data about us and what we’re doing and how we’re doing it. It does so when we are asleep, driving, at our desk, whatever our activity. That data reveals more about us than we know ourselves. It means that organizations can see trends emerging in real-time and it means organizations can tailor products and services to individual needs (a la The Long Tail).

Why do we sell more flowers on Mondays than on Tuesday? Which cities should we stockpile the flu vaccines in this year? Where should I locate the electric car charging stations in London? And the answers are: correlate with Facebook status updates, compare with Google search results of flu symptoms and track the electric car owners paths through the city using the Congestion Charge camera data. Someone’s data contains your answers.

Data is the new oil: extract it, refine it and fuel your business with it.

Posted in Business and Technology | Leave a comment

Internet of Everything Part 2: Nostalgia

Internet of Everything
Nostalgia is so yesterday

In part two we’re looking at the winners and losers in the Internet of Everything (IoE).

Movie theaters will become as anachronistic as libraries are today.
Producers will cut out the supply chain and deliver direct to the consumer’s TV and Laptop and Tablet and Car and keep them synchronized across all platforms by viewer. The better-than-movie-theater-quality-TV-and-sound-system will recognize the user and bring all the content to the screen ready to resume from where it left off. The infrastructure provided by the movie studios is so commoditized already that producers and directors can assemble what they need without the studio.

House of Cards on Netflix never aired on broadcast TV: it went straight to streaming. And there is more to come. I am one of a rapidly increasing segment in the US who does not have a cable provider. What I want to see streams to my iPad/Roku/Apple TV/Chromecast/Amazon Fire. My kids stream to their Xboxes. Cable providers who bundle 300 channels you don’t want with the dozen you actually watch will also go the way of dodo.

National boundaries will become increasingly meaningless because ex-patriots want to see their home TV shows and sporting events live in the language of their home. Restrictions about transmission out of country will disappear as artists and performers realize they have access to a global community.

7 billion people paying 1 cent is the same as 70 million people paying 1 Euro.

Content providers will have more direct connection to content consumers and the networks will be the big losers. Just like the movie studios and theaters.

Content providers will have more direct connection to content consumers and the networks will be the big losers. Just like the movie studios and theaters.

Ereaders and iPads are already the paper of the future. Most people get there news by device already today. Look at any commuter train carriage. The people with the newspapers are the older generation. Everyone else is locked into their tablet reading, listening and watching the news. Even the cross-word puzzle and Sudoku are an interactive online experience.

Print media cannot compete with Internet’s ability to let us see what our friends and colleagues are reading, to cross-reference and correlate stories with live information, make a multi-media experience.

A colleague said to me only this morning, “I used to love to read the newspaper but, by the time I pick it up now, I already know how the story turned out from my Google news feed.”

Newspapers and books will also continue as a piece of yesteryear, a living incarnation of a distant memory. And there is something to be said for the feel of a book and smell of the newspaper but it is only nostalgia that keeps us going there.

I’m sure, a century ago, the horse and buggy were seen as a technology that would soon pass into memory as the technological wizardry of the automobile began to dominate. We still take horse and buggy rides, they are not gone completely, but it is for special occasions and for the nostalgia.

Posted in Business and Technology | Leave a comment

Internet of Everything: Part 1: Kitchen Politics

Internet of Everything
Change brings opportunity, opportunity brings profit

Check out the excellent post here on what is next for the Internet of Everything at PurpleWifi.

In the US, there is a fear that technology is eroding our civil liberties and powerful forces are at work trying to stifle innovation. Soon we will see that small countries, with more progressive societies and legislatures with a willingness to do something, will start to pull away from the US and Northern Europe in the Internet of Everything (IoE).

The IoE is changing how we see the world and interact with it. It continues to improve our lives and makes possible our wildest dreams.

Where there is change there is opportunity, where there is opportunity there is profit.

Far from infringing our civil liberties the IoE give us unprecedented access to information making it possible for us to choose a path in life that is both more fulfilling and more productive. Our ability to be more informed about the politics and the economics that govern our lives is unprecedented. And it will get more so.

There will be some change that people will resist. Full employment may be something we have to forget as a goal. Wealth may have to be redefined and redistributed so our societies can grow culturally, artistically and educationally instead of materially. Contribution to society may have to be measured in new ways that are not based on currencies.

Every walk of life (and even death) already has an app. In the IoE these apps join forces and assist each other in their tasks. The boundaries of the Internet expand to embrace more of our lives. Even the parts of our lives we thought were devoid of technology.

In the kitchen the refrigerator and the pantry will talk to doctor’s office and will recommend the dinner menu based on what ingredients are available and what the doctor has to say about one’s dietary needs. The fridge and the pantry will collaborate and order the food to be delivered so that we never again run out of milk. The oven will pre-heat in time for your arrival home which it will calculate by tracking your location on your commute. By reviewing your calendar the kitchen will know when Aunt May is coming to stay and order in her gluten-free flour and special brand of syrup for the pancakes she always makes. Even the dogs will be fed when your working late without you needing to worry. Irregularities in water and electrical consumption will be monitored and the plumber and electrician called before an appliance fails. When you hold a dinner party the menu planning app will know about everyone attending and their food preferences and allergy concerns.

Even politics is at risk. National boundaries, especially in Europe, are almost meaningless today. National governments provide a 19th century, centralized solution to the needs of a society where communication was slow and the population largely uneducated. In the 21st century communication is instant and all the world’s knowledge is in the palm of our hand. Why do we still insist on these arbitrary lines on maps that do not reflect who we are but who we were. Why do we need representative government when we can all vote by phone on the issues of the day as they happen? Why do we continue to place our faith in politicians who spend half of their time running for election and the half of their time raising the money to run for election. When do they ever do the business of the state. All politics is local, said Tip O’Neill, former Speaker of the House, in the future that can be literally true.

Basques can vote on Basque issues, Kurds on Kurdish ones irrespective of what the lines on the map say.

What will politicians do when the influence money is channeled to the voters instead? Will lobbyists be another anachronism that we look back at with nostalgia?

More nostalgia tomorrow.

Posted in Business and Technology | Leave a comment