2013 NCWIT Summit - Plenary II, Educational Disruptions by Ben Eater

August 14, 2013


Valerie Taylor:  Now, I'll take the opportunity to introduce Ben Eater. Ben is the lead exercise developer at Khan Academy, which is a nonprofit education platform dedicated to providing a free world class education for anyone anywhere. He leads a team of several developers who together are responsible for creating all of the interactive exercises and assessment content for Khan Academy. Ben joined the Khan Academy as a volunteer. After six months, he joined the team as a full time employee. With that, I welcome Ben Eater.


Ben Eater:  Hi, very excited to be here. Just to give you a quick introduction to what Khan Academy is, Khan Academy was started by Sal Khan, who was a hedge fund analyst in Boston who had a cousin in New Orleans who was having difficulty in math. She was in middle school, had been placed into a slow math program, and he offered to tutor her remotely over the Internet. And had started tutoring her and was actually able to get her to the point where she was ahead of her class. He did this by creating some interactive software for her to use, and working with her over the phone. Once word got around in the family that free tutoring was going on, he started tutoring a number of his relatives.

Basically, it got to the point where he was working with 20 or so different relatives all over the country, tutoring them in math and having some difficulty getting this to scale.

A friend of his suggested, "Why don't you put some videos on YouTube?" He's like, "No, no, no. YouTube is for cats playing the piano. It's not for serious mathematics." Eventually, he kind of got over the idea that wasn't his idea and put some videos up on YouTube. The feedback that he got from his family members was that they preferred him on YouTube, than in person.

Which sounds a little back handed, but it actually makes a lot of sense. On YouTube, they could pause the video, they could rewind it. If they were stuck, they could go back and watch previous videos. They didn't have to feel embarrassed that they were asking for him to explain something over and over again. They could kind of do this in the privacy of their own home.

Something interesting happened, the videos kind of ended up taking off. They were getting thousands, tens of thousands, hundreds of thousands of views on YouTube. People started commenting on YouTube, thanking him for these videos. They took a life of their own, and he became just more and more obsessed with creating these videos.

Eventually, he convinced his wife to quit the hedge fund and do this full‑time, not really having a plan for where it would go, and spend about nine months doing this, funding it himself just out of his own savings.

Thankfully, about nine months into this, a number of people started noticing this. Most key among those is Bill Gates, who apparently discovered Khan Academy and was using it with his kids, and was starting to talk about it publicly. Meetings were set up with the Gates Foundation, and Gates funded Khan Academy, turned it into a real organization, and started hiring a team to build up that initial software platform.

But just to give you a flavor of what these videos look like, I'm just going to show a quick montage showing some of the videos that Sal made, as well as we have some more video content creators that have created videos as well.

[video begins]

Man 1:  We can integrate over the surface, and the notation usually is a capital sigma...All these interactions are just due to gravity over interstellar, almost you could call it intergalactic...This animal's fossils are only found in this area of South America, a nice, clean bend here...Notice this is an aldehyde and it's an alcohol... Of course is their 30 million plus the $20 million from the American manufacturer to create the Committee of Public Safety, which sounds like a very nice committee.

Man 2:  This is not Eve.

Woman 1:  No, but Botticelli has portrayed the ancient goddess of love...

Man 2:  This is 6 x 6 x 6, or 216.

LeBron:  I'm told the humidity makes it feel hotter. Why is this?

Man 3:  Excellent question, LeBron. [laughter]

Woman 2:  Let's just make it. Love it.

Man 1:  Play with the pendulum. You get a feel for how it moves.

Man 4:  Function as a bridge rectifier.

Woman 3:  One, two, three, four, five, six, seven, eight.

Man 1:  If this does not blow your mind, then you have no emotion. [video ends]


Ben:  You can always judge the intelligence of the crowd by how much they appreciate Euler's identity. You guys have passed that, I guess. Where we are today, Khan Academy now has scaled. We've had over 85 million users visit the website today, over 260 million videos viewed. That interactive software that I talked about that Sal started using with his cousins, we've now had over 1.1 billion problems solved with that software and we're continuing to grow.

There are over 29,000 classrooms that are using us either formally or informally as part of their curriculum around the world. Very exciting, where this is going. But I just want to step back for a moment and just give a little bit of a back story about how I came to Khan Academy, because it was an interesting start.

I actually never did particularly well in school. I was a C student most of the way through high school and always particularly struggled with mathematics which was always frustrating to me because I was always very interested in figuring things out. I was interested in programming and all of these things that used math but the content and classes never appealed to me or I never got it or understood.

This got even worse when I got to college when I got into a computer science program and the first class that I had to take in college was calculus. I'd never seen calculus in high school because back in middle school, I was tracked into a remedial math program. And so, I never made it to that point.

When I got to college, I got to calculus and I ended up getting a D in the class. Again, I didn't quite understand it. I actually felt like I understood calculus. It's one of the science fair projects I did in high school.

I was interested in computer speech recognition. I was doing some signal processing where I was taking way forms in and doing [inaudible 06:34] and then trying to do pattern matching with Markov models and all the rest. I felt like I had some understanding of the concepts of calculus. I was able to actually apply them, to do things I wanted to do that were interesting to me but I got to the class and it just didn't make sense to me.

It was very confusing. I got a D in calculus which is a passing grade so I moved on to Calculus 2 the next semester, of course, and predictably failed that.

At this point, I thought maybe I ought to go back and take Calculus 1 again and see if I can try this again. My third semester I took Calculus 1. I actually failed it that time as well. At this point, the university and I came to a mutual decision to part ways which is to say they kicked me out and I was OK with that at that point. I was actually very fortunate in a sense that the timing worked out very well for me.

This was at the peak of the dot‑com bubble. It was actually very easy for someone without much in a way of a credential to get a tech job which is what I enjoy doing and so that's what I did. I started learning [inaudible 07:40] in the real world and actually really thrive there and was able to build a very successful career as a result in the networking industry and went on to work as an engineer in ISP and then a large equipment manufacturer and so on.

Finally, I got to the point of my career where I was like, OK, the next step is for me to start a business and so that's what I did. I started a business building some network analysis equipment. I ran into a problem which was that in selling my stuff to my customers, my customers didn't quite understand the benefits of my product and I was trying to explain it to them.

It turned out these are all engineers that I'm selling to. It had been awhile since they had taken their statistics classes. I found myself trying to explain Poisson processes to them and having some difficulty because I wasn't explaining it very well. I went online and said, "Are there some resources that might help me explain this better to my customers so that they understand what it is my product does?"

I stumbled on some YouTube videos from Khan Academy. I'm like, "These are amazing." This explains stuff very clearly. I found the website and I found thousands and thousands of these videos. I got to thinking...I've always been frustrated that school never made sense to me. I feel like in some sense the system failed me.

I felt I was a smart person, but somehow the material didn't connect with me. Here I am 20 years later, let me go back and see where I went wrong. Maybe I can reengage with this calculus content on Khan Academy. This guy seems to speak to me. Let me see how this works out.

I started watching calculus content and I found the same things. There were things I didn't quite understand. They have trigonometry content. Let me go check that out. There were some things I got stuck there, too. Let me try the algebra stuff.

I went back and when you get on Khan Academy, we have this we call knowledge map. Each of these nodes is a concept and it starts at the top with basic addition, 1 + 1, and you can move your way down. It works just like a video game. You master one concept and you move on to the next.

This is the way video games were. This is the way you learn in martial art. This is the way you'd learn a musical instrument. You master one concept before moving on to the next. What I did was I started at the top. I said, "OK, give me basic addition, 1 + 1. I moved very quickly through some of these things, but then I found that there were these concepts really from middle school that somehow I missed. I realized that I had these gaps in my knowledge.

When I went back and went through this, I ended up completing all of these skills. I finally realized why I hit a wall in calculus, which was there were all these little gaps in my knowledge that had just compounded over time. The way that the system worked, there was no way to go back and fill in those gaps.

The analogy we use is imagine you're building a house. You hire a contractor and you say, "The state has given us two weeks to build a foundation. Do whatever you can in two weeks." The contractor builds the foundation. The inspector comes in, in two weeks. The inspector says "It's not quite dry over here. There are some cracks over here. This is maybe an 85 percent foundation."

That's a B. Great, build the first floor. You hire another contractor. You say "Don't worry about the foundation. I need you to build the first floor. Spend two weeks on it. Doesn't matter if the supplies are late, doesn't matter if it rains. Just do what you can in two weeks."

Contractor goes, builds the first floor. Two weeks later, the inspector comes in. The inspector says "This isn't quite up to code. You forgot to build one of the walls. We'll call this a 65 percent first floor." You say "Great. That's a passing grade, build the second floor."

This is exactly what was happening to me in my education. You build floor after floor after floor. You get to the fourth or fifth floor, the whole building collapses and everyone wants to blame the contractor. Or they say "Oh, we need more inspections. We need better inspections." But no, the inspections were fine. The inspections were identifying these gaps. The system, the process, didn't allow us to go back and fix those gaps.

In my experience, I was going through a lot of these exercises on Khan Academy and these are some examples of what those exercises look like. This one is sort of a Montessori for calculus, allow you to get the intuition for a derivative.

I went through all these things and I felt very passionate about what Khan Academy was doing. I discovered that actually this exercise platform for creating these things was open source. In my spare time, I started building some of these. All the ones I'm showing you are things that I built as part of that open source project.

I started to send these things in and I got positive feedback from Khan Academy. I got an email that says "Would you like a job?" I was like "No, I have my own business. But I really love what you guys are doing. Keep it up."

A couple of weeks later, I got another email with "Here's a bunch of statistics on how many people have been using your exercises and thousands of people have. Do you want a job?" I was like "No, I got my own business. I'm happy. Things are going well." Then Sal called me a couple of weeks later and was like "Do you want a job?" I was like "No."

At that point, they were very clever. They said "We feel like we should be paying for your work. Why don't we set you up with a contract? We realize you're busy. You have this business. We'll set you up with a contract. You don't have to work at all. Or you can work as much or as little as you want, do whatever you want and just bill us."

I was like "OK, I can't turn that down." I turned down the job a few times, but I can't turn that down. I went ahead and started doing contract work and just over the course of a few months, I just became more and more obsessed. I found I couldn't help but spend more and more time creating these exercises for Khan Academy.

I had a bit of a decision to make. I had this business that was reasonably successful. Everyone was telling me it would be ridiculous for me to leave my business and go work for a nonprofit. I've been a capitalist my whole life. Everyone was telling me this.

I guess I had to make this decision. Eventually, I ended up at a party with some friends that I hadn't seen in a while that knew me in college and knew what I went through. They're "Oh, of course, you should do this." I guess that was the validation I needed and so I picked up, moved across the country and joined Khan Academy. That's what got me to Khan Academy.

As I was showing you these exercises, this thing keeps going forward too many things. Once I got to Khan Academy, one of the things that was really kind of mind blowing for me was not so much...I had my own personal experience of going through and realizing that I had all these gaps in my knowledge and that if I went at my own pace, I could fill in these gaps and the videos were there to help me.

But what was really mind blowing was how Khan Academy was being used in classrooms. One of the things that we're doing is we go into a classroom, we start to give teachers these analytics. Basically, what this chart is showing is...This is a graph that we give to teachers. The rows are students. The columns are each of those concepts that we saw on that knowledge map earlier.

Green means a student is proficient, blue means a student is still working on it, and red means a student is stuck.

What the teacher can do is the teacher can walk around the classroom. This thing updates in real time. Students are all working at their own pace at their computers. The teacher sees someone who turns red, maybe this student is struggling in exponents three. There's another student there who is proficient in exponents three and the teacher can start to set up a peer tutoring arrangement there.

If that doesn't work out, the teacher can intervene one‑on‑one. Maybe the teacher can see "Hey, a bunch of students have mastered this concept, so now we can do a project together as a class that involves them," knowing that all the students are going to get something out of that.

One of the other dashboards that we give teachers, we have a bunch of different dashboards that teachers have access to if their students are using Khan Academy in the classroom, is this. What this is showing is along the bottom is the number of days that a student has been working on the site and the Y axis is the number of skills that that student has completed. Each of these lines is a different student.

What you see when you start off is that some students race ahead. They get in. They really engage. They get a bunch of skills. Some students are in the middle. Then some students, the flatter lines there, are a little slower. Those are the students that you might say "Oh, these are the remedial students. These are students we're going to pull out of class. We're going to put them into these remedial and track them separately."

But what we find, time and time again, and you can never identify these students ahead of time, is that if you let everyone work at their own pace, some of those students that you thought were remedial, they might struggle on a concept for a little while, but once they get unstuck on that concept, if you let them work at their own pace, they race ahead.

That highlighted example of a student there is an example of one of those. It turns out that she ended up as the first in her class. We see this time and time again.

This is great for the math concepts. One of the other things that we wanted to push is how can we express creativity, or how can we use creativity in them. The way that we do this is we came up with this computer science platform. A lot of people say, "I was asking about creativity and the creative writing and that sort of thing. How does computer science relate to that?"

Of course, you guys know, computer science is fundamentally creative, computer programming. We created this platform that was designed to inspire creativity. This is just an example. You see the code on the left is creating this animation that you see on the right. You can update the code in real time. If I change that number, you see the little rainbow thing changes shape.

You can change it to a different number, now they get bigger. You can even drag the number around and you change the size. You see what you're doing in real time and so, you get a real sense of what the code is doing.

We have a bunch of these different examples that people can work through. What you can do is if you make changes to this, you can save your copy as a spinoff and it shows up underneath the program. You can see these are all just different things that different users have created and gone in their own direction.

They've gone in and explored the code, explored, "Hey, what does this variable do if I change this? What does this do?" They've come up with all these different things, and they can create a spin‑off. Then that shows up on their profile. This is what someone made of that.

Of course, you can make your own things from scratch, as well. If you get stuck we have a series of videos that work within this platform. I'll just show you an example of one of them to give you a sense of what that looks like.

[video begins]

Jessica Lew:  Here's where the magic comes in. Somewhere in the draw loop, we're going to change the value of x to be a little more than it used to be. Like this, x gets the old value of x plus, let's say, one. Yay, it works. Except, [laughs] it's so smeary. If you're wondering why it looks that way, it's because we forgot to draw this background inside the draw loop. It's drawing the car over and over again, but you can see all the old cars underneath the new one.

If I pull this line into the top of the draw loop, like that. Then press restart so I can see my car again. Yay. It's so perfect.

If we want to make the car go faster, we can change how much we increase x by every time. If we make it 10, it's off the screen.

[video ends]

Ben:  This is one example. We have a whole bunch that talked obviously about animation. We have a bunch of these videos that were all made by Jessica Lew, who was a phenomenal intern that was with us last summer. We hope we will see her again. But it just gives you an example of some of the things you can do. All of this, all of the code, everything, runs within a browser. That's the other great thing in terms of the testability to this. You don't have to set up a whole dev environment to get started.

You point your browser to Khan Academy and you can start writing code. Or, you can start pulling one of the examples that we have and start making your own spin‑off from that.

It, really, inspires creativity. There are questions, comments, tips, feedback and everything that goes underneath there so you can collaborate with other people. You can have comments. People can tell you, "Hey, did you try this or think of this?" So very exciting.

To wrap up, one of the things you may have noticed is everything I've been talking about, so far, has been in the English language or focused on the US. One of the really exciting things that's been happening, is there have been a number of different groups, non‑profits mostly, some governments, that have been taking Khan Academy and taking it around the world.

These pictures are some examples of Khan Academy being used around the world. One that really excites us is in the top left corner, there. We used to joke, "Hey. Maybe someday Khan Academy will be used in Mongolia." [laughs]

Not too long later, we got an email from Mongolia. It was from this girl Zaya. She sent in a little YouTube video thanking us for Khan Academy.

We assumed she has access to YouTube, the Internet, Khan Academy, she was probably a middle class girl and all that. We read the email a little bit more. It turns out that, actually, there's this group of engineers in Silicon Valley that on their vacation, they go out to Mongolia. They're setting up computer labs in orphanages.

That picture in the top right are Mongolian girls in a Mongolian orphanage using Khan Academy. Zaya was one of those orphans.

If that's not cool enough, Zaya got really into it and she started translating some of the videos into the Mongolian language. She's 17 years old now, and so this 17‑year‑old Mongolian girl is now the prime producer of Khan Academy content in the Mongolian language. She's become the teacher for her country.

To give you a flavor of what these videos look like, I'll show you another quick montage. It ends with a piece of that original video that we got from Zaya.

[video begins]

[foreign words in various languages]

Zaya:  I am Zaya from Mongolia. Your videos are so interesting and can you make more lessons? [video ends]

Ben:  We watch that whenever we're feeling a little lazy around the office. [laughter]

Ben:  One of the amazing things is our team, right now, we have 40 people. Over the last year, we averaged about 34 people. With the scale of the Internet, we were able to reach over 50 million unique students in 216 different countries. I really think we're at this Gutenberg moment. Not even a once in a lifetime moment, really, once in a millennium moment, almost, to push the scale of this forward. The original tag line on this was, "With just the necessities."


Ben:  To wrap up, education has always been seen as this scarce resource and this thing that was very expensive. This thing that separated the haves from the have‑nots. Even when people are doing philanthropy, they would say, "What do the rich have? That's expensive. Let's create a cheap approximation of that and we'll give that to the poor." The poor had nothing before so a cheap approximation is better than nothing. But now, education is becoming no longer a scarce resource.

We're almost getting to the point where, I honestly believe, education is going to become a fundamental right in 20, 30 years. People are going to view education as a fundamental right, just like clean drinking water, or shelter or electricity, something like that.

That's all I had. I thank you all.



Transcription by CastingWords