jason (FMA Admin)

As entries pour in for our remix contest with the Prelinger Archives, we thought you might be interested to learn more about where the digitized portion of the Prelinger collection is housed, the Internet Archive. Brewster Kahle, the founder and digital librarian of The Internet Archive (archive.org), was our guest on Radio Free Culture June 25th, 2012. He was interviewed by Ken Freedman, WFMU's station manager.

Radio Free Culture is a series on WFMU and the Free Music Archive where we focus on digital culture and issues like net neutrality and piracy, digital rights, archives, libraries in the internet, all with WFMU’s own particular digital viewpoint. The following is a transcript of the program which can be heard over here.

Ken: Welcome to "Radio Free Culture." I’m very, very excited to have with me. Let’s start off by introducing the thing that you do which is, for lack of a better way, trying to get a copy of just about every piece of information in the world.  Is that right?

Brewster:  Yeah.  The idea is to try to build the Library of Alexandria, Version 2.  You can basically build a world where if anybody’s curious enough [they] can have access to the books, music, video, anything that’s ever been produced, anywhere, anytime.  And I think that’s one of the promises of the internet and the Internet Archive is trying to play a role in that.

Ken:  And have you surpassed The Library of Alexandria?  In terms of how much information it had?

Brewster:  Well they say that they had 300,000 scrolls back in 300 BC but it was about 75% of all of the books ever written, whether it was in Egyptian, or Greek, or the Hebrews or the Hittites, whatever. I wouldn’t say we have 75% so I think we’re still being aced out by original Library of Alexandria.  

But we do have something going for us is we can actually get these materials to people no matter where they are.  So you don’t have to be friends with the Ptolemies to be able to have access to the collections either on the Internet Archive or the internet in general.  But, man it’s been sorta quirky in terms of all the rights issues; the technology issues have actually been working out pretty well.

Ken:  How much information is on the Internet Archive?

Brewster:  The Internet Archive, let’s see, it’s got about 8 petabytes of data now.  It goes mega- giga- tera- peta-.  So we’re at about 8 petabytes.  We get about 2 million users a day, different people that are using the collections.  So it’s encouraging that people actually want old stuff.  We’ve got about a half million people using The Wayback Machine, which is a collection of pages from web sites we’ve collected that since 1996.

Ken:  The Way Back Machine is a big source of embarrassment for me here at WFMU because you have pages from our web site from  1996 which are downright embarrassing.

Brewster:  Yeah, well, just take a look at M.I.T.’s if you ever want to feel better about yourself.  In general things looked a little hokey back then.  It’s nice to be able to see the evolution of the web, but it’s also nice to be able to hold some people accountable.  You know, go back and see either their privacy policies or whatever their offers were and be able to go and say “Hey, wait a minute!  You said this!” and we find that often people try to go back to their web site and just sort of wipe things off and think that it’s gone.  The average life of a web page is about 100 days before it’s either changed or it disappears. The Wayback Machine now has 150 billion web pages from over a hundred million, a couple hundred million web sites.

Ken:  Brewster, when a new service comes out on the internet like Twitter do you sit down and start strategizing how you’re going to start copying every single tweet that goes out?

Brewster:  Yeah, we’ve tried but we have missed, something we’re not proud of.  We get the popular things with Twitter.  And it’s actually how we learn of the YouTube videos that we want to go and archive as we just watch Twitter for all YouTube links and then go and archive those.  

We also missed Napster.  Just completely.  Maybe one of the best music libraries ever built, which was burned by the stroke of a pen of a judge and we weren’t on the ball.  So there are whole services that we’ve missed.  But we’ve been trying to do what we can.  We’ve been working pretty hard at keep up with the web, now even some software, there are books we’re very active in.  We’ve been archiving television for about 10 years.  

Ken:  When I was telling my wife about what I was going to be talking with you about, she had a naïve but kind of profound question, which is “Why are you doing this?  Why are you trying to archive the entire world and the entire internet?”  

Brewster:  It’s really based on an analogy, if you will: that it seemed to work in other times - that the idea of having access to the collective works of humankind has been a win.  So, we all look back to the Library of Alexandria.  And by going and pulling together the works from all over the world and translating them then into Ancient Greek, they were able to come up with fantastic discoveries.  They knew how big the world was.  They knew it was round.  They knew how big it was within a couple percent.  Euclid authored “Elements,” which is what it is I still studied as geometry in high school.  So, fantastic things can come of it if you can leverage the works of other people.  And the reason why I got involved in the whole area of building the library back in 1980 was just kind of on that analogy and the thought that technology allows us to do this and it seems like a good thing to do.  

But there’s another step that’s kind of amazing is that it’s all intermingled now with computers.  So there are now computers and the world’s literature, and everything else kind of all jumbled up together.  And I went to school and technical school and studied artificial intelligence, and the idea of an artificial intelligence is quite doable.  It may not be quite as separate as we imagined it then where they’ll be these separate borgs over there and we’ll have to go and fight with them, but it’s now much more intermingled.  I think of Google as sort of an interesting organization:  it’s mostly machines.  It’s machines with content.  And so we now have this combination which is much more popular and much more powerful than just having books on shelves.  So that idea was in the air, sort of, in the 1980s and I said “oh, I dunno, let’s go build that” and so far so good.

Ken:  And you actually began building The Internet Archive in the ‘80s.

Brewster: Well, building a lot of the earlier technologies.  First we needed computers that could help search things.  So, [take a] guess, computers then were pretty terrible.  So I helped build supercomputers.  We made them so they could search materials and then made it so you could remotely connect to them.  That was the early ‘90s.  Then the world wide web came up later.  So I helped by building the first internet publishing system and probably the first internet search engine.  So anyway, once we got the publishing going online which was sorta by ’94-’95, then we could turn to build the library.  So that’s when we started the archive: in ’96.  

Ken: It seems to me that having a gigantic library like The Internet Archive is a form of artificial intelligence.  And Alan Turing, the great mathematician and theorist postulated that artificial intelligence will be achieved when a human being can converse with a computer and not know that he or she is conversing with a computer.  That’s the idea of the Turing Test.  Do you accept that as the only definition of artificial intelligence?

Brewster:  I think it’s an interesting test. It’s sort of like, you know, can computers beat us at chess? And now, check.  Going and defeating the top human at Jeopardy! I thought was a stunning achievement.  But I guess I’m not quite looking for the cyborgs walking around and looking sexy, I’d just like them to be kind of interesting and useful to be around.  And I’d say we’re already there.  I certainly look at my kids and they’re plugged into screens and interacting with other people and other computers.  And having it be a pretty smooth interface, that makes it hard to tell a lot of the time exactly why is it you’re attracted.  Is it the person on the other side?  Yes.  Is it the technology?  Yes.  Is it the built up knowledge that is built into these systems? Yes.  We’re building kind of a cyborg-human mush.  

Ken:  Okay we have the first question from somebody listening on our comments board.  “the glowing one” wants to know “what are their strategies (for The Internet Archive) on data rot and data migration?”

Brewster:  Ah, good questions.  How do you deal with it – if you can actually collect this stuff, then what do you do with it?  And how do you keep it going?  I think there are a few different areas.  There’s just trying to keep the materials such that the bits that you have today, you’ll have tomorrow.  And that’s by, basically, making copies.  We have to move it onto new media every 3 to 5 years.  And so we’ve been doing that, um, for the last couple decades.  So that actually is kind of doable as long as you’ve got continuity in your organizational structures.  So that’s one way.  

Then there’s trying to get it so that it’s accessible.  So, how do you go and deal with old formats?  And you have to just keep moving it forward.  But the thing that I’ve really found out in this is that you really need a lot of copies.  You need multiple copies.  If the Library of Alexandria had gone and made a copy and put it into India or China, we’d have the other works of Aristotle, the other plays of Euripides.  Which we don’t.  And it was because they only had one copy and the idea of universal knowledge kind of faded from being popular as the Dark Ages set in.  So we have now made partial copies and put them in the New Library of Alexandria in Egypt which, by the way, is one of the most beautiful places to visit ever.  Highly recommended.  And also in Amsterdam.  

So, we now have partial copies in a couple different places.  If there are 5 or 6 places with active organizations going and keeping them going, and also there are large-scale swap agreements, then I can feel safe.  We’re also starting out some experiments with Bit-Torrent to sort of see can it help, not only in access but in preservation.  So we can get more of a distributed infrastructure that doesn’t depend on large organizations.  What happens to libraries, over time, is that they’re burned.  And they’re generally burned by governments.  Just, historically, that’s not a political statement, it’s just historically true.  So, let’s design for it.  Let’s go and make sure there are copies in different regions that are on different political rampages at different times and I think that we have an opportunity to at least build things that will last for centuries.  

Ken:   Brewster, you’ve been there on the internet since long before there was a world wide web. Can you talk about how the original intentions and the original designs of the internet have been changed, and I would think for the most part for the worse?

Brewster:  Well, I got involved in it from the university angle, and from that perspective we saw it as a government program.  So ARPANET, going to the Internet, was sort of military-oriented, sort of research and educational networks.  And then it had an interesting merging, at least for me.  So I helped build a bunch of those sorts of things in the late 1980s, then I met up with the guys that came at it from the modem perspective, much more human-oriented.  

There was Mark Graham who started PeaceNet, Mitra (Ardron) who started GreenNet in England.  And these were people that were trying to hook people to together to be able to get political things done.  It was a very different angle from how the university side was going on and they developed these whole systems of communities and how to communicate.  And so there was a merging of those worlds, sort of, in the 1990s, first few years of the 1990s.  The World Wide Web helped things a great deal, just to make it so you could point and click and move around.  So you didn’t have to have expensive technologies, rather clunky technologies that came before that.  

Then there’s that wild commercialization thing that happened in the late ‘90s.  It was 1992, I was in a meeting at Stanford University and I held up my hand as was like “I’m here to help people make money by being on the Internet,” and that was like “ohhh no, are you allowed to do that?!?” and that was sort of as far as things went in 1992.  And now, I sit in rooms and say “I’m here to help people actually have a public life on the internet that isn’t mediated just by commercial interests.”  So I think we’ve swung the pendulum too far.  Fortunately there is a role and the growing nature of things like Wikipedia or The Internet Archive or Linux or EFF, Public Knowledge, Public Library of Science.  We’re now starting to get these non-profit organizations that are really the infrastructure organizations.  We’re able to build operating systems.  The number 5 most popular web site is Wikipedia.  That’s pretty darn impressive.  And it’s all non-profit.  I have a lot of faith in this sort of secular non-profit structures that we’re building up around technology.

Ken:  I guess what I was getting at in terms of the architecture being spoiled are the massive “walled gardens” that we have on the internet.  Such as Facebook and the I-Tunes Store.  How does archive.org get at these “walled gardens”?

Brewster:  Let’s just build an alternative that’s much more fun and interesting.  When we had AOL vs the World Wide Web that was also sort of “walled garden” vs “wild west” and frankly, just being on the “wild west” was just more fun.  It was just more interesting.  That was where the happening stuff was going on.  It wasn’t contained and controlled.  But we have to keep the open-end environment so that it doesn’t feel like it’s just spam-filled.  Email is just starting to feel like, it’s just a drag.  And that’s why people are retreating towards systems that are kind of moderated and controlled by single corporations.  That’s all-around scary. 

Also, we’re starting to see things like the cell-phone world and Apple starting to really give up on the general purpose platform.  It all has to come from them.  That’s not going to come up with the next generation of really great applications if it has to be OK’d by the incumbents.  So having a bit of a wild and wooly world, keeping it fun and interesting, is something that we see at The Internet Archive as absolutely critical to not only the survival of our own organization, but sort of what it is we’re here to help foster.  So, we help support things like EFF and Wikipedia and we use Ubuntu.  I do use Apple products, but they’re really kind of going a little closed.  

Ken:  What do you think about the rumors going around that Apple is ultimately going to eliminate browsers from their platform?

Brewster:  Browsers are starting to just get embedded and blended in with their worlds.  I haven’t heard that particular rumor.  I just think we’ve got to keep focusing on the open world.  We can’t slam the door such that the next guy just can’t come along and make something better than we did.  The idea of going and saying “well, you know, alright, we have all the apps we need.  We’re done,” is just not the way we should make things go.  It’s not the way to have an intellectual history that grows and changes and is full of the new ideas that are needed.  Clearly the generation that’s in charge right now is not doing that great of a job.  So let’s make sure that there are some openings toward doing something a little differently, and hopefully a whole lot better.

Ken:  Brewster, you’re on the steering committee for the new Digital Public Library of America project.  Can you tell us what that’s all about? 

Brewster:  Yes.  There’s a group of libraries that were working with Google to digitize a lot of books.  And there was sort of a counter project which was one that I helped run towards making openness.  The Google libraries got tangled up in courts and got stopped in the courts.  And there’s sort of this question of “okay, what do we do now?” among them.  Some of them got together and started the Digital Public Library of America and they’re trying to learn from some of the lessons of the Google project.  And we’ll see sort of what happens out of that particular project. 

But I think a lot of countries are doing things that are definitely in the open domain, which I’m very, very excited about.  Actually China, for one, is digitizing a lot of books within the Department of Education, making these available, all within the legal constraints, to make things open and available.  There are about 500 members of the Open Content Alliance that are trying to build open libraries that don’t have centralized points of control.  So I think it’s too early to tell what’s going to happen on the Digitial Public Library of America.  

There are good things happening in the books realm around lending, digital lending, of scanned materials.  So books that are digitized from the 20th century, that aren’t the current best-sellers, are being lent using the same constriction technologies that the publishers use for their in-print books.  But these are then lent and borrowed for 2 weeks and then it comes back.  So there’s only 1 person at a time that can have these old dusty-musties.  But it is bringing access now to hundreds of thousands of books out there on the net.  So there are ways of working within the library metaphor, if you will, to do format shifting of some of the older materials into the newer technologies so you can read them on your iPads or iPhones so if you go to openlibrary.org, which is a website that we built and run, there are 10,000 books for everybody, and then in a lot of libraries, like Boston Public Library and a lot of other libraries, if you go in, there are 200,000 books that are available.  So I recommend seeing how that’s evolving as a mechanism of trying to deal with balancing the interests between commerce and access to information.

Ken:  We have a question from listener Bas in the Netherlands.  He wants to know is there a way that you place value on what goes into the Internet Archive?  Is a value judgment made or do you just try to get everything across the board?

Brewster:  We really try to get everything across the board.  But we do have to go and point our machines and engines in different directions.  So we’ve been trying towards the popular or the ephemeral.  So we started with the World Wide Web, because it was the most transient.   And then we started archiving television, 20 channels of television at DVD quality.  Russian, Chinese, Japanese, Iraqi, Al Jazeera, BBC, CNN, NBC, FOX.  But, you know, 20 channels wasn’t everything.  We’ve now upped it to around 100 channels.  

Ken:  So you are archiving 100 channels or worldwide TV 24 hours-a-day? 

Brewster:  Yes.  We can’t make it terribly available yet.  And we’re trying to figure out how to get it more available.  We did September 11th through September 17th and that’s up on the archive.org: archive.org/911 . It’s a little hard to watch, but it’s interesting to see how did the Palestinians react? How did the Russians react?  There was this theory that, there were some American TV stations saying that the Palestinians were dancing in the streets.  Well were they?  Well, you turn to Palestinian television and it turns out you have a very different point of view.  

So I think we now understand that news comes with a point of view.  And it’s not one really brought home in the United States.  But, of course, those that watched really closely of course knew this forever.  But I think it’s now wide-spreadly accepted.  So if we’re going to think critically about this major cultural thing, which is television, then we’re going to want to be able to quote it and compare and contrast.  And that’s very difficult to do.  So, that’s what we learned in high school on how to write a good essay but you can’t really do that.   You can do that with the newspaper, but you can’t do that with television.  So we’d like to try and get that to happen.  

Ken:  I guess, speaking of the ephemeral nature of the internet, I guess the elephant in the room might be pornography, which is, I guess, a great driver of commerce and technology online.  And there’s a huge amount of pornography sites that come and go.  Do you make any attempt to grab any of those?

Brewster:  Yes, but we only get the free stuff.  So we haven’t gone and subscribed and then gone and archived the other sections.  Pornography is -what an interesting role it plays in society, but it’s often been sort of under the table.  I talked with one of the leaders, former leaders, of the Library of Congress:  he said, “You know, the stuff just gets stolen.”  It gets submitted because it’s got copyright deposit requirements into the Library of Congress, but it just… goes away.  It was required that people send in a couple copies of the CDs that used their software.  This was accumulated, this is during the 1980s and ‘90s, it would accumulate.  But the pornography would just get kind of stolen.

Ken:  So you’re saying that the copies of Hustler and Swank in the Library of Congress are walking.

Brewster:  Well, I can’t say that. At least some of it, some of it.  It’s just kind of interesting.  Anyway, so we’re doing what we can on the publicly available materials.  But we haven’t done a concerted real effort in that particular genre.  T

here are other areas that we have taken a real effort like the work of Rick Prelinger to archive ephemeral films: old educational films, advertisements, industrial films, those sorts of things.  And he’s now moved to –we’ve worked with him and we’ve digitized a lot of those films, put them up on archive.org.  And they’re fantastically popular.  They’re downloaded hundreds of thousands, sometimes millions, of times for the most popular of these things.  And I’m not really quite sure why.  Maybe with people it’s kitsch, maybe they’re remaking things into their own movies - all sorts of reasons.  But he’s now moving into home movies.  So, the idea of taking all of these old films that are raw documentation of life at a different time is interesting to people, not just your family.  So I think there are areas of ephemeral materials that we’re pretty good at.  

Ken: Now what are some of the legal concepts that allow the Prelinger archives to be freely available?  I know that, for one thing, that if government produced a film then that is freely available as public domain.  But are there other such concepts like that?

Brewster:  Oh, boy.  Copyright stuff is just endlessly complicated and really contorted.  I’m not the best person to ask.  There are certainly areas that are made available and it’s not a problem.  So things that are before 1923 had to register copyrights between 1923 and 1964 and then reregister 28 years later.  It’s sort of a hole of incantations.  

But that said, you have to take things to appeals courts to be able to figure out if some things are copyrighted or not.  So there’s an evolution of this, what are really the protected areas that we want to make sure stay clear of commerce.  And then there are other things that are sort of moving forward.  You think of YouTube and there’s all sorts of stuff on YouTube and there are just more and more deals going on to figure out what are the right lines for this internet age?  We don’t want to interfere with those that are really concerned about particular things.  For instance, on the Wayback Machine we archive people’s web sites.  But a lot of these web sites weren’t really designed for forever and so people either put a “robot exclusion” on their website, which we then respect retroactively or they write to archive.org - info@archive.org and we’ll take things out of the Wayback Machine.  And that type of respectful, quick, dealing with the issues seems to be working out quite well.

Ken:  What are the reasons that most people would prefer their older pages to be taken off of the Wayback Machine?

Brewster:   Often they’re embarrassed by what things might have been on there, time to move on, I think is some of what we hear anecdotally.  Things that they just didn’t really intend to last the ages.  If people go to The Internet Archive, archive.org, and they type in their old college web page, that they built up their blog or whatever, often there is sort of this “WOW!  Oooooh..” Sort of “that’s really cool, it’s there!” or it’s not there and then there’s “Wow.  Am I better off or worse off because it’s there?”  And we hope most people feel like they’re better off because it’s there, because it has some level of relevance to the future.  But, other people come up “Aahhhh, nahhhh, let’s take it off.”  So we take those things off of the Wayback Machine.  

Ken:  So since I have you on the line now, can you remove all WFMU pages from prior to 2002?

Brewster:  We don’t know how to do a time-based thing, so I think your embarrassing old web pages are just going to have to stay up there.  

Ken:  Okay.  Now you mentioned the Google digitizing project that ran into legal problems.  I’m not sure what you’re referring to.  Were those the orphaned works issues, where Google was just trying to go out there and just digitize every orphaned work that they could find?  

Brewster:  They’ve digitized about, they’ve said, 12 million books in these great libraries working with these libraries.  And for the public domain books they’ve added contractual restrictions.  So you can download them onesy, twosy, but you can’t download a bunch.  If you wanted a thousand books of Hungarian origin because you wanted to do some research project, they’d turn you off.  This is also enforced by the libraries that they’re working with.  So that’s what they’ve done in the public domain which we think is really atrocious.  

Public domain is small enough as it is.  It should be fully available and completely.  So when The Internet Archive digitizes things with its 500 partners, we digitize about a thousand books every day.  And if they’re public domain then they go out to everybody with no restrictions at all.  You can download all of them we’ve digitized.  There are now over 2 million books on archive.org and about 10 million books a month are downloaded.  

So anyway, public domain should just be everywhere and they’ve decided to something differently.  Then there’s the question of, “what do you do with the things that have rights issues, that are rights burdened?”  And so there was this approach towards building a centralized organization called the Books Rights Registry and the Hathi Trust, which would be the ones that sort of owned and controlled these books.  There might be other Hathi Trusts allowed by the Books Rights Registry, but it would all be centrally administered.  And that’s just not an American way to go.  Going and giving an organization that much power over the history of literature just isn’t really how things work here.  And so it was objected to and the courts said, “naah, you’ve got to do something differently than that.”  So we’re really organized toward having many winners.  

We want to have many publishers, many libraries, many authors who make money.  And everyone is a reader.  So the idea of having lots of winners and no central points of control is how we got here on the internet.  It’s why Creative Commons is kind of an interesting approach, why open source has been working well.  The world wide web doesn’t have central points of control.  This is just the way to make a robust, evolving, environment.  So we’re resistant to any of these approaches.

Ken:  So did Creative Commons play a really big role in the evolution of The Internet Archive or was it simply an idea?  

Brewster:  It was huge.  It was huge.  When we first talking with people after we just did the world wide web, and moving that along, people wanted to go and put things up on The Internet Archive.  And so they said “well, send me a contract” and it would just be this bogged down legal rubbish and we couldn’t get through more than 2 or 3 of these a year.  It was awful.  

But when the Creative Commons license came out we just said “Look, there’s a Creative Commons license, just pick one.  If you pick one, we’ll put it up.”  And we have, within our public funding, and our foundation non-profit funding we know we can support it.  And it sped things up.  Now we deal with hundreds of organizations and it works great.  So that structure for mechanisms, of just sort of reeling back, it’s trying to basically come before 1976 Copyright Act in the United States, which was really just one of the worst ideas coming out of the United States in the 20th Century, where they made everything copyrighted, and almost forever.  Before that, you actually had to go put a © on things and send a copy into the Library of Congress to get protection.  But they flipped that around.  And Creative Commons is trying to bring us back to the – look there are just some things we want to go and protect and sell and whatever but mostly, you know, let’s have it out there.

Ken:  One of the things I really love about the Internet Archive, and what I really admire about what you are doing, is the way that you’ve jumped over all the digital rights and copyright issues that still seem to snag and ruin almost every other conversation about archives and librarians.  When did you figure out that copyright is the wrong framework for talking about these things?

Brewster:  Copyright is just part of the arsenal that people have to try to stop things they don’t like.  The key thing I learned by doing the Wayback Machine was people are, when people see the stuff up there they’re trying to figure out are they being taken advantage of?  If they feel like they’re being taken advantage of, they’ll figure out some way to cause you trouble.  Copyright is just one arrow in that quiver.  

There are all sorts of different things you can do to try to tangle up an organization or somebody else that you think is doing you wrong.  So the key thing is to try to stay such that you don’t piss people off.  And so being respectful, I guess, is probably the biggest.  

And one of the wonders is actually out of the music world.  There was a tradition that was started by the Grateful Dead which was they allowed their fans to tape their concerts and trade their concert recordings.  This has gone on years and years before and I had my cassettes back in the ‘70s from Grateful Dead concerts and you just sort of knew who the top Deadheads were by how big their collection was and how few generations they were from the master copies or whatever.  And we had an intern working for The Internet Archive (that was in 2002) and he said “You know, tape trading still exists and it’s just moved on to the internet.”  I said “Nooo” and he said “Yeah, yeah, yeah, there are lots of other bands that do it.”   And so I said “Okay, well why don’t you offer them unlimited storage and unlimited bandwidth forever for free?”  So we went and he wrote to the eTree community which was sort of the group that administered tape traders and said we’d be up for hosting these materials, unlimited storage, unlimited bandwidth, for free.  And somebody from that community wrote back and said “We don’t believe you.  It’s too big.  But if you could do it, it would be our dream.”  Which is a really good sign.  So we said okay, well let’s try it.  And we did it - it was a little different from tape-trading to go and hosting  on a web site so let’s get somebody within the band or somebody within that community to say it’s okay. 

And so it wasn’t in signed triplicate, which is what your lawyers would all say to do, it was somebody within the band or something saying it’s okay to host this on the archive.org.  And if anytime you want to take it back down again, we’ll take it back down again.  So we started getting 3 bands a day signing up with this and about 40 concert recordings being uploaded to archive.org and it’s been going on now for 10 years.   We just crossed over 100,000 concert recordings and 5,000 bands have signed up.  And these are fantastic.  It’s everywhere from bands that are playing in real venues – some of them are signed artists and some of them aren’t.  And it’s working.  

So you take that whole period of time when there were all these lawsuits going and suing grandmothers and kids - just dreadful – and we’re finding that there’s a path through this.  There are ways to make it so that everybody’s happy:  the library, we’re happy with it, the fans are happy with it, the bands are happy with it.  It seems to be all around working.  And I think we just need to kind of walk through some of these doors, maybe a little slowly, saying what we’re trying to do, how we’re trying to do it.  We’re not making a lot of money off of this stuff.  In fact we’re making no money off of this stuff.  And that was key to the Grateful Dead.  And that seems to help get through things that might just be tangled up for years if we didn’t do it that way

Ken:  I have a vague memory of at a certain point the Grateful Dead actually took a mild amount of issue with all the tapes that were on archive.org and wasn’t a compromise worked out?

Brewster:  Yes.  Oh gosh, that was a tough couple of weeks.  We had been contacted by one of the Grateful Dead archivists who knew that we were doing this, but there was some internal talk within the band community and they said “Brewster, you’re gonna have to take it down.”  And it was great – there was no legal thing, no “cease and desist” – it was sort of all how you would imagine the Grateful Dead would do things.  And they said “you’re gonna have to take this down.”   And I said “I’ll take it down, but it’s gonna cause a storm.”  And they said “uh, sorry you’re still gonna have to take it down.”  

So there are two classes of materials in the Grateful Dead collection: there are audience recordings, which were really what was supposed to be traded and there are soundboard recordings, which sort of leaked out of the vaults but they were collected up.  And then we put them all up on The Internet Archive.  So we have, well, everything the Grateful Dead has ever done.  

So these were out there and then we took them down.  And the world exploded.  It was in the New York Times, it was on CNN twice.   And then there was just this very public dialogue, back and forth, about what should happen to all of this.  And there was a compromise that was reached.  And I think that, you may disagree with it, but what happened was the audience recordings are all publicly downloadable, so you can go and download those and trade those as long as nobody makes any money, and the soundboards are streamable.  And once that compromise was reached, the whole thing died back down again.  

Frankly I think we’re on firmer footing because the grandfathers of the sharing world sort of weighed in and had a public debate as to where do you draw the lines on these sorts of cultural artifacts that weren’t being exploited commercially?  Oh we would get contacted and they’d say “Hey, we’re coming out with another ‘Dick’s Picks’ – which is another concert that we’re gonna start selling.”  And then we’d pull it down from The Internet Archive.  Again, we don’t want to interrupt commerce.  We just want universal access to all knowledge.  Some of it will be for pay, some of it will be for free.  But at least let’s make sure that it’s up and preserved and available.

Ken:  Now we have a question from listener Curtis.  He would like to know whether there are facilities for archiving data for public access via archive.org? His question is “Are there facilities for archiving data for public access via archive.org?” 

Brewster:  Hit the button “Upload” in the upper right of the site and you can upload any file.  If it gets super popular, usually it means then we go and look at it and sometimes it’s something that we don’t think we should be having on The Internet Archive.  Or sometimes people go and flag things and say “that’s kind of inappropriate.”  But, yeah, it’s basically free hosting of books, music, video.  You can put data – please don’t put crypto-data or things like that, it’s not meant to be a back-up of your hard drive.  But things that you think would belong in a library.  We want all cultural artifacts.  So, please do upload.  Oh!  We’ve got a beta-version , a different version, if you go to archive.org/upload , I think it’s a better interface.  It hasn’t really been publicly available yet, but check it out.

Ken:  Now I guess by allowing anybody to go onto archive.org and upload whatever they want, you come in under the safety of the “Safe Harbor Provision.”  Is that accurate?

Brewster:  Yeah, there’s the Digital Millennium Copyright Act which is general is a dreadful piece of legislation but they did make it so that there’s a sort of a “notice of take down” provision that if users go and put things up on these web-sites then the websites are not responsible unless they are contacted by rights-holders and say take it down.  And that’s really made the internet and user-generated content possible.  

There is also the Communications Decency Act, Section 230, that is out that really helps.  And that’s why we have the internet that we’ve come to enjoy today.  There’s an attack on that, actually, by the Washington State Legislature.  They wanted to basically make it so, for certain types of content, advertising sex turned out to be the thing that they used in this circumstance but we’ve seen other things where – kiddy porn or terrorist whatever.  But anyway, so this time around they were using that as the area to make it so that web sites would be criminally responsible for what their users put up on their web sites.  So the EFF and The Internet Archive have sued to try to get that law blocked. Currently, it’s, the judge basically said the law could not go into effect yet until there are more hearings.  So we’re encouraged.

Ken:   Now what have you learned from overseeing The Internet Archive that you’re going to bring to the Digital Public Library of America?  

Brewster:  I’m not sure what’s going to happen exactly with the Digital Public Library of America.  But if we take the more general role of trying to build the Library of Everything I think we need to flex and be respectful.  We need to really work to get some of these other communities on line, like art videos.  Art videos: they’ve never been very available.  You know you’d see them in these fun and interesting movie theaters once and then they’d sort of go back away.  So how do we go and get that stuff online?  

How do we get music so that it’s better representative so it’s not all just going through a couple of single gatekeepers?  Even when they’re sort of as big and strong and capable as iTunes and Spotify, I don’t think that’s really where a lot of the future of music is going to come from.  Books.  Journals.  I think the open journals are really doing very well.  Public Library of Science is showing that you can get tenure by publishing in open-access journals.  And that’s really changing how academia works to get it back to where it used to be, which colleges were more or less copyright-free zones.  And that, I think, can be restored by these open access movements.  So I’m encouraged by many of these movements that are going on from much smaller organizations than sort of the big libraries, big publishers.  We’re seeing a lot of creativity come from below

Ken:  And we have a caller.  Chris is on the line. 

Chris:  Brewster thank you very much for what you’ve done over the years. Without you I would not be listening to my “CBS Radio Mystery Theater” or “Bob and Ray” or Jean Shepherd.  But there’s another thing.  I was checking out your FAQ and it says that normally if you want to upload something you have to have some sort of like license or copyright or some sort of ownership of it?  With the upload itself – a few months ago they nuked Megaupload and I used to use that as a source for sharing Bob Lassiter files with people, and that’s no longer currently available.  Would I be able to take, like, open source mp3s of Bob Lassiter and upload them to the archive?

Brewster:  As long as you don’t think that we’ll get in trouble for it, absolutely.  I don’t know who Bob Lassiter is.  So if it’s owned by somebody then don’t get us in trouble if you can.  

Ken:  Yeah, my experience, Chris, has been that most talk show hosts are happy to have their stuff distributed in that way.  So if you can just find somebody, somebody from his family that says it’s okay, then I’d just go ahead and do it.

Chris:  Okay.  Okay, thanks Ken.

Ken: Thank you Chris.

Brewster:  In general, it is an evolving standard as to exactly how screwed down should all of this be.   Certainly if you can get somebody, even if they’re not going to say in triplicate, again, it’s okay.  But if there’s somebody from within different communities, whether as you said Bob Lassiter’s family or the like, this is sort of the way we deal with rock and roll bands.  If it’s not something you’ve made then try to make sure it’s not going to be offensive to those that might take offense.

Ken:  Now you mentioned Spotify before.  There’s been quite a debate raging about Spotify since it pays probably the smallest amount of royalties of any legal music streaming service and Spotify’s argument is that it’s better than piracy.  How do you weigh in on that kind of thing?

Brewster:  Wow, I just recently went and tried out Spotify.  I sort of did the, okay, does it have the obscure stuff?  And it seemed to have everything that I could think of.  It was pretty darned impressive.  So my hat is off to them in that way.  They seem to have just a tremendously deep collection.  Which I’m really quite happy that it’s as accessible as what they’re doing.  But what should happen?  I don’t really know.  I think that we should have music libraries.  Real libraries within the library traditions that also have collections that are as deep and as broad as Spotify.  

Ken:  What are some of the challenges facing the future of The Internet Archive as well as the DPLA?  What are the things that might bring you down in the future?  

Brewster:  What will bring The Internet Archive down?  I think it’ll be a growing irrelevance to the sort of deep materials that The Internet Archive really focuses on.  That I think, either that or a stroke of a pen.  

I think libraries are burned by governments and the way that they burn them now is with the stroke of a pen.  And we’re seeing whole websites, whole organizations, get taken down with one swath, based on potentially a minority of the files that are on them.  So SOPA and PIPA, which were those failed pieces of legislation, had that kind of “Let’s strike down whole websites at a time – not just individual pieces of materials.”  And that I find frightening.  So unless we can do really strong things to go and protect the open internet, it will disappear.  It wasn’t around when I was growing up.  To be able to get distribution you’d have to fight your way through retail chains and all sorts of awful things.  

We’ve built up something that’s been amazingly open.  But it’s gonna require absolute defense.  I would support, and I’m a board member of, the Electronic Frontier Foundation.  I think they’re doing one of the best jobs in some of this.  And be active and go out and protest when it actually comes time to do it.  

Ken:  Were you impressed with the outcry over SOPA and PIPA when that took place?

Brewster:  I was so pleased, really.  Jimmy Wales going and leading the charge to darken Wikipedia, I think, was one of the biggest issues.  So I think there are ways that we can win this one.  The corporate influence on the legislature at this point, I think we’re just going to get bad laws after bad laws.  So I think we have enough laws that we need now, that we don’t need new laws in many of these circumstances.  ‘Cause the laws as they come up are just worse each time, at least from the public’s point of view.  It might benefit some corporation, or they might think it might, but it often doesn’t play into the public interest.  So I think we’ve got the rules, the laws we need.  We’ve got the civic responsibilities to build libraries that are robust and serve the public good.  We’d just like more them.  We’d like one of these in every country.  

Ken:  Well that’s a great way to end it.  

Brewster:  Thank you very much, Ken.

This series is supported in part by an award from the National Endowment for the Arts.