Iceland review - 2007, Qupperneq 21

ICELAND REVIEW 27
interview
Alëx Elliott: Tell me about the revolutionary algorithm you created.
Kristinn Sigurdsson: Well, it’s a little bit complicated actually: I creat ed
two algorithms. The one that’s being used is actually a fairly simple way
of detecting duplicate data. So what we do is we comb the web – we send
out our little ‘spider’ that gets everything, and the one I wrote realizes
what’s changed and what hasn’t. And when it detects something that
hasn’t changed, it simply doesn’t save it.
AE: What are you actually collecting?
KS: Pretty much all the stuff under the Icelandic domain, .is.
AE: And what about irrelevant stuff that’s on .is, like foreign websites
and blogs?
KS: Well there’s not much of that because registering a .is domain is
relatively expensive compared to a .com or .net, so people don’t gener-
ally buy them. You’ll notice the occasional one – just during our last
crawl, we ran into a porn site that basically buys every possible domain
for its name, so it has .com, .net, .is, .co.uk, whatever they can get their
hands on.
AE: As you mention porn: are there many porn sites that get collected?
KS: There aren’t many porn sites under .is, but if they are there, we try
to collect them. We try to not exclude anything unless we know it to
be banned. Typically, even if it is porn, it is still content and we would
want to archive it because we don’t really know what future historians
are going to be interested in.
AE: Well, quite. And on the subject of the validity of storing electronic
information: is it reasonable to expect that electronic information can
last a thousand years, like the sagas did for example?
KS: Well, if you know your Icelandic history, the sagas were in pretty
poor shape by the time they were rescued and started to be maintained
sensibly. It’s a lot more diff icult to maintain and preserve digital data
than paper, just because when ten years pass and nobody’s paying
attention, it can all be lost. You can’t just stuff it in a vault some-
where and cross your fingers. That said, I think the big problem is
going to be formats because if you have a document format now, like
Microsoft Word, no problem, you can open it anywhere. But if you
have a Word Perfect document from 20 years ago, you might have a
lot of problems. If we look into the future, is anyone going to be able
to open a Microsoft Word document in 50 years?
AE: Is it a good thing to be collecting everything? Because a lot of it
is presumably not going to be relevant.
KS: Well, if you compare it to our regular legal (paper) deposits, we
collect everything: that’s the law.
AE: But there is a certain standard to published work though, isn’t
there?
KS: Well yes, there’s some truth to that. If it’s published on paper, at
least somebody has to pay for it. But sometimes when they’re looking
into historical documents, the things they find most interesting are
the things that weren’t archived in a structured manner: stuff that just
happened to survive. Those are often the most interesting, probably be-
cause they’re so scarce. So, if we do start cutting stuff out, somebody’s
probably going to be mad at us at some point because that’s exactly what
they were interested in. Our position has been that we just don’t know
which material might be irrelevant.
AE: If you just do a normal Google search, you can get things from
ten or more years ago, so you must be looking in the very long term
with this project?
KS: We are definitely looking in the long term. We are the government
institution responsible for preserving cultural heritage in the written
word, and as such, how long will the government be interested in
doing that? We like to think forever. I mean even if the Internet goes
away, this data might still be relevant. Presumably it (the Internet)
would be replaced by something better in some sense. I guess if we have a
technological collapse we might be in trouble, but that’s not something
I think we can guard against effectively.
AE: How did you get involved in the project and what does it mean
to you? I mean, is it fun for example?
KS: I joined the library in early 2003 and initially I wasn’t supposed to
do this; somebody that was already here was doing it. However, it
turned out that there was this project to build the collecting tool
Heritrix at the Internet Archive in California. The Scandinavian count-
ri es came up with this crazy idea to send two people out there for six
months to help out and to push the things we wanted. I turned out, being
single and an expert on Java, to be one of the most suitable candi dates
they had on staff. Before that trip I didn’t know much about Internet
archiving, but after the trip I guess I was the leading expert here. For
me, archiving the worldwide web, as a project, is of more or less just
academic interest. But as a professional challenge, it has been totally fascin-
ating.
AE: And what about the future: the project’s going to continue, but what
about you?
KS: To be honest, I f ind this work to be exciting enough and extremely
challenging. It would take something really interesting to get me to change
my views on that.

Part of Kristinn Sigurdsson’s job is to collect everything that is ever put on the Internet on any website ending in ‘.is’. He is the IT
group project manager at the National Library of Iceland, and the special algorithm he helped create makes the task a
little easier, but Alëx Elliott wanted to find out why they are doing it at all.
Copy and Paste for Posterity
Photo by Páll StefánSSon