The following is an excerpt and case from one of the classes I teach in Crisis Informatics.
This is a story that relates to this and is one of my favorite experiences. I went to a cooking class with my wife in Varenna, Italy. We were staying in Bellagio and took the ferry over across Lake Como and then took a taxi cab up to this restaurant on the top of the hill, over Varenna where we met our host. I called him Professor Marino, because he was the chef who was teaching this class and this day he happened to be teaching in English. He taught it in certainly in Italian, and may have taught it in French also. But I called him Professor Marino because he was a bit of a philosopher, as you'll see. So, the group that was there for the class, we were told we were going to make gnocchi. So, the first question, we asked him after he took and he poured a whole bag of flour onto the table. And he made like a volcano shape out of it. And he put four eggs in the center. And then he starts mixing it with his fingers. And he's mixing the dough and mixing the dough. So, the first question was, how can you tell if the dough is mixed enough? And he said, close your eyes and touch the dough. Think of something beautiful and touch the dough. And you'll know when it's ready. Of course, if you've done it many times before you know when it feels right, because you've been there. And the next thing he does is he starts rolling out the dough with his rolling pin. And then we asked the next question, Well, how do you know when the dough is thin enough? And he says, you can see the grain of the table, the grain of the wood will show through the dough and that's how you know that it's thin enough. Now, he said, in Italian cooking, for most dishes, you need to taste and adjust, taste and adjust, taste and adjust. It's an inexact method. And he said that's true for most every Italian dish you make, with one exception. Italian pastry. For Italian pastry, he said, you must follow the recipe, you need to have the ingredients exact, you need to have the temperature in the oven exact, you need to have the time that it's in the oven exact, otherwise you ruin the pastry.
So, an interesting question to consider, and applying this as a metaphor: is most of business, including technology, about making pasta, or pastry? [dramatic pause]. It's about making pasta. In other words, it's handling ambiguity. So, if you prefer to work with a pastry approach and have an exact recipe, you're going to get very frustrated in your business career. Learning how and developing strategies for dealing with ambiguity will help you navigate that transition. Things at the university tend to be fairly exact. Rubrics are fairly exact. Exams are exact. But work that we do in corporations, in nonprofit organizations, and governments tends to be rather inexact. And there's a fair amount of discovery, experimentation and exploring, that you just plain have to figure out. That's a challenge.
So, why is it important to be able to handle ambiguity? Well, everybody in leadership expects you to be able to handle the ambiguous situation. And ambiguous projects, especially when they're projects from clients. These have a greater probability of being ambiguous.
Your initial encounter with client ambiguity may come during your interview for the job. I'll illustrate this with the Amazon Interview case, below, where the candidate was deliberately asked ambiguous questions to see how they would handle it, to see what their thought process was, how they broke it down, and made sense out of it. Because when you're doing client projects, that's the thought process you're going to need to follow. It's also an opportunity to prove your value-add at times. How do you make sense out of the chaos? Can you demonstrate that?
The Amazon Interview Case
A question was posted on Quora.com, presumably in Dec. 2016 [3], by a developer who had applied to Amazon and did not pass the on-site interview. They asked Quora readers for advice about what to do next. One of the responders was Matt Kellner, a software engineer from the Seattle area [4]. 1.5 million people have read his answer to-date.Matt indicated in the follow-up comments that he previously worked for Amazon, and that he has been on both sides of the interview table. He notes that the line of questioning where an interviewee “locked up” is typical at tech companies. It’s deliberately ambiguous. It creates a mini-crisis to see how the applicant handles it. Read his response in full, below.
Also read the other replies, as there are interesting perspectives on tough interview questions. The bottom line is that companies are looking for candidates who can handle ambiguity. It’s not only for solving open-ended problems internal to the organization; often customers do not know exactly what they want and are not particularly adept at providing focused requirements. In fact, one could argue that in the realm of customers, whether internal or external, perfect requirements don’t exist. So handling ambiguity is fundamental to tech-related work.
Here are the discussion questions:
- Have you encountered ambiguous questions during interviews? How have you handled them?
- What do you think about Matt’s advice on how to handle ambiguous questions? What resonates and what doesn’t?
- If the goal is to have a successful job interview (i.e., generate an offer of employment), how will you handle the inevitable ambiguous or open-ended question?
- What do you find frustrating about this, and how do you propose to deal with that?
- If a goal of crisis preparedness is to build more resiliency, or the ability to “bounce back”, how will you prepare for the interview crisis?
Here is the Quora.com excerpt:
What should I do after failing at
Amazon's on-site interview?
This question previously had details. They are
now in a comment.
11 Answers
Matt
Kellner,
Software Engineer, Video Game Historian and Part-Time Farmer
“I wouldn’t call you a “loser”[5], but it does sound like you did one thing that interviewers don’t like to see in technical interviews. When you get a technical question in an interview, that question is almost always going to start out vague - it’ll have some important pieces missing such that you actually CAN’T adequately solve the problem without more info.
In a case like that, the interviewer is
being very deliberate about the ambiguity. They may be simulating a request
from a customer where the customer knows what they want but didn’t do a good
job of communicating it. The point here is to see how you handle ambiguity: Do
you make assumptions? Do you lock up? Do you go down a path toward a solution
without understanding the question? Or do you try to identify what you don’t
know and start asking questions to clarify?
I’ve interviewed a lot of people who
did what you described: They just stood there staring at my question, not
talking at all, not letting in on their thought process, etc. The problem is,
you might be thinking through the problem, but the interviewer can’t tell what
you’re doing, so to them, it looks like you just lock up.
We generally understand that people get
stage fright in some situations. The interview is a stressful, grueling process
for most people, so it’s not surprising that you’d have a case where you just
lock up - some people don’t even realize they’re doing it. That’s why a good
interviewer will try to prompt you, will ask you to describe what you’re
thinking, will give you small bits of guidance, etc.. We WANT you to succeed,
and it’s in our best interest to do whatever we can to help you succeed.
But there’s only so much we can do. If
it becomes obvious that you’re completely lost on a problem, or you have to be
hand-held through the whole thing, or you’re just not going to “perform” no
matter how much help we give, that’s not a good sign of how you’ll do in a real
work environment. The real work environment, especially at Amazon, is often
more stressful and faster-moving, and your success or failure on a real problem
could make the difference between success and failure of the product you’re
working on.
So, here’s what I always recommend to
interview candidates:
·
Brush up on your problem-solving
skills. Remember that most interviews are about how well you can understand and
solve problems, not just how good your code looks.
·
Always, in some way, repeat back the
question you’re asked. It’s up to you to decide how best to do this - you can
paraphrase the question, you can draw a diagram on the board, you can describe
it in terms of behaviors, etc. But make it clear that you do (or do not)
understand what the interviewer is asking you to do.
·
If you find a point where you CANNOT
do that, ask the interviewer to clarify. For example, if I ask you to divide
two numbers, and you realize that I haven’t specified what should happen when
the second number is zero, just ask me - something like “What should this
method do when I try to divide by zero? Throw an exception? What kind of
exception should we throw?” etc.
·
When the interviewer clarifies a
question like that, you might then realize you have even more questions.
Explore those. Take notes on the board as you get these clarifications, as they
will inform your solution.
·
Keep things moving. Keep talking,
think out loud, draw, write, whatever you need to do to keep your interviewer
informed as to your thought process.
·
Finally, don’t worry about finishing
your solution. Most of the time, it’s not about actually writing and fully
testing a finished piece of code - if you get there, that’s great, and a good
interviewer will probably add a bit of complexity, quiz you on testing or on
the operational complexity of the solution, etc.. But most of the time, they’re
more interested in seeing how you approach problems, how you think and how
willing you are to reach out for help. (This includes cases where you know what
you want to do but don’t necessarily know the correct syntax to do it in
whatever language you’re working in. If you can describe what you’re doing, the
interviewer can usually play technical reference for you.)
If it helps any, I’ve done a number of
interviews (on both sides - being interviewed and giving the interview) where
we never got around to writing a single line of code. Instead, we spent the
whole time just discussing the problem. :) I personally think those kinds of
interviews are the most fun.
Hope this helps. :) Last piece of
advice: Don’t be afraid to apply at Amazon again in the future. You’re unlikely
to have been “blacklisted” - let some time pass (say, 6 months or a year),
brush up on your skills, and if you’re still interested in trying again, apply
to new positions. I can’t guarantee that they WILL interview you again, but
they certainly might.
… I remember that
someone in the comments took exception to where I said “Instead, we spent the
whole time just discussing the problem”, as it seemed (to that person) that I
was saying interviews really are only about conversations and not technical. I
want to clarify this point:
What I meant there was that we
sometimes ask (or are asked) questions that are really huge in nature. For
example, in one of my own job interviews, I was asked how I would design a
banking system that could handle deposits, withdrawals and account transfers.
When I started exploring edge cases, concurrent users, data storage mechanisms,
account locking and fraud detection, etc., the scope of the problem blew up to
enormous proportions. What started as a simple exercise in defining some common-sense
interfaces for these individual functions turned into a wide-ranging discussion
that touched on both general banking concepts and on distributed systems
design. Given that where I work is concerned with such things, this was
on-point.
Once I was hired, the person who did
that part of my interview mentioned that he fully intended for the discussion
to grow like that, as the “test” he was giving me was on whether I would just
try to solve the direct problem, or if I would start looking at the “big picture”.
I’ve found in many cases that this is one primary difference between “junior”
and “senior” engineers - since I was applying for a senior position, this level
of discussion was appropriate.
My point here is to illustrate what is
meant by some interviews not involving coding, even though you may be
interviewing for a coding position. Software engineering, particularly the type
that Amazon does, entails a lot more than just basic problem solving and
algorithms. As you move up the chain, you can’t avoid taking on more in the way
of large-scale design, testing, project management and other things that you
may not be used to doing. And these interviews, in part, are done to see how
ready you are to step outside your comfort zone.
(Update 2, April 8, 2018) I was reviewing
this question and found another answer I hadn’t seen before, stating that you
should be happy that you didn’t get hired at Amazon, and linking to a Google
site documenting “management abuse” and such. I’d just like to say that, as a
former employee of Amazon, I strongly disagree with this person’s answer (they
disabled comments, so I can’t respond directly). While I cannot speak for all
departments at Amazon, I worked in a total of four of them while I was there,
most of that in AWS, and my experience there was nothing like the majority of
reports on that site.
There are a couple of groups at Amazon
that most people agree are less than ideal to work for, but my experience was
largely positive, and the difficulties I encountered had more to do with the
“frugality” aspect of Amazon’s culture - most notably in running “lean” teams
that do more work per person than at most other companies. But I was proud to
be a part of the groups I worked for - I did some of the best work of my career
(so far) in those groups - and I left the company on good terms and would
consider going back in the future.
I won’t tell you not to check out that
“FACE of Amazon” site, but keep in mind, it appears they have posted only
universally negative stories there, and thus the site seems to suffer from
confirmation bias. Their language and their use of an anonymous email system
suggests that they are not interested in positive stories for balance. If
you’re concerned about what it would be like to work at Amazon, I would encourage
you to look also at sites like Glassdoor to view ratings from a more representative sample of
current and former employees, as well as by job type and title.
In my experience, Amazon has very high standards
for its employees, and it CAN and WILL burn you out if you let it.
However, they do also respect you when you tell them what your limits are - if
you can still do your job, you should not fear letting your manager know that
you can only work 40 hours a week.”
About the
Author
Software Engineer, Video Game
Historian and Part-Time Farmer
Software Engineer1997-present
Studied at California Polytechnic State University, San Luis
Obispo
Lives in Puget Sound
[3] The earliest comment is from Dec. 8, 2016, after expanding all the comments,
here: https://www.quora.com/What-should-I-do-after-failing-at-Amazons-on-site-interview
. The original question posting does not appear to be currently available on quora.com.
[4] https://www.quora.com/What-should-I-do-after-failing-at-Amazons-on-site-interview/answer/Matt-Kellner-4, Last
Accessed on August 15, 2018
[5] See the comment by Heck Evergreen, Senior Software Dev Engineer in 3/10 Top Companies (2005-present), answered Jun 4, 2017
