5:56 pm
·
The tale of what happens when it gets close to closing time
This is the story of every weekday spent
looking at the top right,
My Macbook pro, slowed down by a million
tabs in plain sight,
There used to be buffered videos and
downloaded documents,
Now there are bookmarks, there is
pocket, yet the same predicaments,
At my old job, this time meant quitting
came easy around now,
We’ll pick this issue up
tomorrow, someway, somehow,
At best this will be a bug, at
worst something that needs a fix,
It isn’t that bad, it’ll be good
in a couple of weeks,
Regress this and run that suite,
lets make sure we follow process,
Lets have a meeting pile on
another, we need our leaders to bless,
But now it’s you, it’s the clock, it’s your
cabin without a view,
You have hours and minutes and
new problems to chew,
You have the same problems, just
a smaller set of people,
You have the same headaches, just
a different source of evil,
What became of team lunches and
morale events?
What happened to planning
committees and coffees over nonsense?
It is but a function of moving to something smaller,
Punishment and learning — the hopes of
becoming a baller,
Relinquishing options for equity
and a moonshot,
Redbull, caffeine and venture
funded food for though,
What determines happiness and keeps
insecurity at bay,
At least here in the bay, dreams
never get in the way,
Plenty of dealers who encourage a
hand,
Plenty of castles made out in the
sand,
Staying lean and floating like
ice,
Untouched vagrancy, brought about
by vice,
In these 4 minutes that the A/C whirs to a
halt,
You look away, analyzing the
lines of your fault,
You head away to recharge your
batteries,
The drive back, drivel on the
radio drown out your inner sighs,
To each their own as you check your phone
stuck n traffic,
Eating out again, since cooking
stopped giving you the kick,
Is this really a worthy tradeoff
for work and your sanity?
Nothing better to throw yourself
at and no one for clarity,
Finding your calling has little
to do with a number,
But you have been clueless for as
long as you can remember
Thug Life
·
What does it say when I cant hit chrome again,
How can I be so blase when there is so much to gain?
A lifetime of struggle punctuated by battle,
A multitude of users herded like cattle,
So when they click, and make that call,
I can survive and pretend to stall,
The ever close ceiling that will make me wonder,
The gravity of motion, that will come asunder,
One more line of code, and another feature,
So I can breathe life into this creature,
That can mint money when I press run,
But leaves me broken, unable to face the sun,
Pizzas and Beer, declining health,
Questionable social skills, already shrouded with stealth,
Fatter than ever before and no set schedule,
A nerdy T-shirt, messenger bags that look cool,
And yet i rise, late again,
Motivation tipping and demands insane,
A missed call and errors of my ways,
Why has this deal not been closed in days?
Set targets and bonuses to be spent,
You wonder which way hope went,
But it is a thug life that you signed up for,
Take me back to school, I dont want this war,
Sleepy eyes and timecards to boot,
Putting out fires, lungs stained with soot,
You live to fight
And in spite of in spite,
You keep the faith and soldier on,
Little knowledge and fear of the long con,
This hope, that ebbs and flows,
Keeps me sane when the wind blows,
Happy hours that bond with colleagues,
Revolving doors and warehouse digs,
Its a template that has fooled so many,
Its a punchline that seemed so zany,
But feet up and a commit made,
The race starts again until you are dead,
Is this the life that you really wanted?
Or was it a nightmare that came unsaid?
The addiction fuels your fight for another 2 weeks,
Your games are over when you play for keeps,
Try as you might, this has you hooked more than drugs,
But this life is here, replete with thugs
How can I be so blase when there is so much to gain?
A lifetime of struggle punctuated by battle,
A multitude of users herded like cattle,
So when they click, and make that call,
I can survive and pretend to stall,
The ever close ceiling that will make me wonder,
The gravity of motion, that will come asunder,
One more line of code, and another feature,
So I can breathe life into this creature,
That can mint money when I press run,
But leaves me broken, unable to face the sun,
Pizzas and Beer, declining health,
Questionable social skills, already shrouded with stealth,
Fatter than ever before and no set schedule,
A nerdy T-shirt, messenger bags that look cool,
And yet i rise, late again,
Motivation tipping and demands insane,
A missed call and errors of my ways,
Why has this deal not been closed in days?
Set targets and bonuses to be spent,
You wonder which way hope went,
But it is a thug life that you signed up for,
Take me back to school, I dont want this war,
Sleepy eyes and timecards to boot,
Putting out fires, lungs stained with soot,
You live to fight
And in spite of in spite,
You keep the faith and soldier on,
Little knowledge and fear of the long con,
This hope, that ebbs and flows,
Keeps me sane when the wind blows,
Happy hours that bond with colleagues,
Revolving doors and warehouse digs,
Its a template that has fooled so many,
Its a punchline that seemed so zany,
But feet up and a commit made,
The race starts again until you are dead,
Is this the life that you really wanted?
Or was it a nightmare that came unsaid?
The addiction fuels your fight for another 2 weeks,
Your games are over when you play for keeps,
Try as you might, this has you hooked more than drugs,
But this life is here, replete with thugs
The Watchers rise
·
Hi
I am Amit, I work at a big tech company, looking to go full time with my idea that I am working on the side. What do you do?
I work for .....
Watch the reactions closely. The crowd will thin gradually. They will gravitate towards the clearly accomplished entrepreneur out here on an ego mission. Some others will move towards the older exec looking to reinvent himself. Yet more will move to the brunette who discusses marketing as the cornerstone of success.
And yet you find yourself, in between conversations with seriously interesting people, generally aimless. If you ever find yourself lost in a crowd that is out to practice the tenets of networking, you are a watcher.
I define Watchers are the tribe of people who are dabbling, they are involved but not committed [obligatory bacon and eggs story]. In exercising my brain cells some more, I find distinct categories of people at networking conventions.
The Doers - They believe they have found their calling. They are self appointed born entrepreneurs. They move heaven and earth to work on their startup. Tons of respect, great hustle, flawed motivation.
The Naysayers - Hands down the most annoying set of people who you will meet. They will shoot down your idea and burn your business plan. They are great critiques but terrible if you want to get a sounding board. Their advice skews negative and having made peace with not ever having a venture of their own, they are the naysayers. Annoying as fuck as I have pointed to before.
The Entrepreneurs - Probably not at this event because they are busy raising money or setting up analytics on their MVP.
The Watchers - Lets go back to the first conversation. Everyone who starts down this path is a watcher. They have a number of interesting characteristics
1. Insecure - Cannot take the plunge of leaving a cushy 100k+ job
2. In need of validation - Not just your vanilla kind of validation whores, these folks will seek ever increasing validation on either their product or idea before embarking on a journey of building
3. Smarts - Worship the Lean Canvas to the extent of drawing it out in the air and pointing at flaws in you business in terms of Alexander Osterwalder
These folks are also in search of activation energy. In advertising jargon, they are well qualified leads. They actively seek out opportunities and validation. They seek out help, partners, potential funding but are crippled by a fear of failure. A fraction convert but not nearly enough. They make up reasons to not succeed or give themselves a reason to buy a 1 way ticket to a cushy job.
Interestingly enough startups like One Month Rails are ever decreasing the threshold of entry for these watchers. But is that a core problem. Youtube videos and tutorials existed prior to Udemy and Coursera. If it is indeed motivation, then TechCrunch and their lavish dreams have you covered. Resources on the internet cannot possibly be a problem.
Why are the conversions failing for the Watchers? Are we even asking the right questions to get more people to kick their jobs to the curb and just try something out? What do you think?
I am Amit, I work at a big tech company, looking to go full time with my idea that I am working on the side. What do you do?
I work for .....
Watch the reactions closely. The crowd will thin gradually. They will gravitate towards the clearly accomplished entrepreneur out here on an ego mission. Some others will move towards the older exec looking to reinvent himself. Yet more will move to the brunette who discusses marketing as the cornerstone of success.
And yet you find yourself, in between conversations with seriously interesting people, generally aimless. If you ever find yourself lost in a crowd that is out to practice the tenets of networking, you are a watcher.
I define Watchers are the tribe of people who are dabbling, they are involved but not committed [obligatory bacon and eggs story]. In exercising my brain cells some more, I find distinct categories of people at networking conventions.
The Doers - They believe they have found their calling. They are self appointed born entrepreneurs. They move heaven and earth to work on their startup. Tons of respect, great hustle, flawed motivation.
The Naysayers - Hands down the most annoying set of people who you will meet. They will shoot down your idea and burn your business plan. They are great critiques but terrible if you want to get a sounding board. Their advice skews negative and having made peace with not ever having a venture of their own, they are the naysayers. Annoying as fuck as I have pointed to before.
The Entrepreneurs - Probably not at this event because they are busy raising money or setting up analytics on their MVP.
The Watchers - Lets go back to the first conversation. Everyone who starts down this path is a watcher. They have a number of interesting characteristics
1. Insecure - Cannot take the plunge of leaving a cushy 100k+ job
2. In need of validation - Not just your vanilla kind of validation whores, these folks will seek ever increasing validation on either their product or idea before embarking on a journey of building
3. Smarts - Worship the Lean Canvas to the extent of drawing it out in the air and pointing at flaws in you business in terms of Alexander Osterwalder
These folks are also in search of activation energy. In advertising jargon, they are well qualified leads. They actively seek out opportunities and validation. They seek out help, partners, potential funding but are crippled by a fear of failure. A fraction convert but not nearly enough. They make up reasons to not succeed or give themselves a reason to buy a 1 way ticket to a cushy job.
Interestingly enough startups like One Month Rails are ever decreasing the threshold of entry for these watchers. But is that a core problem. Youtube videos and tutorials existed prior to Udemy and Coursera. If it is indeed motivation, then TechCrunch and their lavish dreams have you covered. Resources on the internet cannot possibly be a problem.
Why are the conversions failing for the Watchers? Are we even asking the right questions to get more people to kick their jobs to the curb and just try something out? What do you think?
Year in review
·
What have I learnt in the past year?
Last year was a good one. Not in the sense of an average,
curmudgeonly existence, but in the sense of progress and direction. It was a
good year. My most significant year was my 24th. The opportunities
and the open head surgery performed on my thought processes over my time at
Stanford was life altering. My connections there make it very vivid and
memorable for me still. My 25th was a year of reconciliation. To get
back to a profession is underrated. The ability of a human being to keep coming
back for more is what is so underrated. To keep coming back thinking that you
will have something new to learn is persistence. It’s the other side of the
coin to insanity where you try the same thing over and over and expect different
results. It really isn’t about the effort you put in so much as it is about how
you go about it.
Lesson 1 – Persistence has value. Patience probably not
Patience is a construct that is misunderstood often. To be
patient when the other option is being a headless chicken wont help is not
wise, it is sane. To be patient when your options are exhausted and the only
way forward is through someone else’s intervention is not an option, it is all
that is left for you to do. So you never choose patience, you cop out on it. At
least that’s my take.
Lesson 2 – You do not know what you do not know
So many curveballs in the last year. I had an interview with
Palantir that I thought went really well. Didn’t get the job. Was it
disappointing? Yes. It forced me to re-look at my career choices and take some
time off. I was still applying but I had very little conviction. I had patience
but not persistence. Then things changed, a startup and a product later, my
confidence is pretty much where it should be. I never really liked the products
built in and around big data. It seems like a fancy con, and a long one.
Something that scares me. A friend of mine said, big data is just the same as
small data. You don’t have to worship it. When I joined Greenplum (Pivotal), I
was pretty sure that I wanted to write specs. 2 months in, I was working on
products with a team that did only pair programming and took their orders from
the Pivotal Tracker. I thought I would have a hand in open sourcing a
proprietary piece of software. It open sourced itself and I was in the
audience. Months later I thought I would be good with Hadoop and storage.
Which, as unsexy as it is, has gained traction. A good career move I thought to
myself. A month later, I was gone.
Lesson 3 – Uncertainty undermines you.
I wanted to get my Visa stamped in India because, you know,
immigrant. I was scared. I had heard horror stories of RFEs and peeps getting
stuck in India for a long ass time. I wanted to make up for the money spent in
getting my MS over the next year. I did not want to be a slave to the dollar –
rupee machinations. I canceled my flight once because I had a bad vibe. And for
long periods of time, I did not want to go. I finally did and managed to get my
visa stamped. It was fear that made me not get a car here in the US. I was
constantly wondering if I would be stuck on the other side of the Atlantic or
not. I was never gung ho about life. I was uncertain. I was pretty fucked up. I
embraced the fact that I did not know and carried on with life. Take a
decision. Even if its bad, you learn from it. If you deliberate, its going to
be a while before it gets better.
Lesson 4 – Be unapologetic
A lot of my concerns and questions over the years have been
silenced by my fear of failure, and even worse, my fear of having to admit it.
If you have been put in a position where you need to take a decision, take it. Don’t
apologize for it. If they didn’t want you to pull the trigger, they wouldn’t have
given you a gun in the first place. Shit happens, embrace it (metaphorically,
not in a 4Chan kind of a way).
Lesson 5 – If you don’t learn a new thing everyday, you are
undercutting yourself
Knowledge is power. It may not be money.
Trivia is not knowledge. Trivia is suited for pub nights.
Knowledge is when you have a coherent hypothesis around what happened. Trivia
is knowing which year it happened in. Money comes from winning the pub quiz. I
have been told that I must outline everything I know on a document and then get
others on the same page before coding starts. If I had my way, I would shove a
skunk so far down the throat of the maker of this particular request that their
burps could evacuate buildings. Knowledge and learning take time. And it is a
slow process of unraveling layers and mysteries. No body starts off with a
perfect view of the world. We iterate on what we know and if we are lucky, we
get somewhere.
.JPG)
Nuts and Bolts of a payment startup
This weekend, after shuttling between Pawnee and Scranton, I was pretty interested in finding out about the machinations of the newest payment startups. To get to a point that I can even understand about the value proposition of these payment startups, I needed to learn the basics of payment startups. And across the web, after scouring the primers available, I found the following links. Here is a watered down version of the basics, I am sure it would be interesting to people learning about the basics of these startups.
Now I am Adam or Alex or Adnan or Amit or well, lets just call me A.
A wants to use his credit card to buy a new ergonomic keyboard. This Keyboard is listed on buycheapkeyboards. This keyboard is something that A has wanted for a long time but only now does his budget allow him to go get this. A whips out his credit card and orders the keyboard from the website. A fills out all the details like so
After filling out the details, A hits Ok and waits for a few seconds. Now what happens in those few seconds.
After the credit card details (Name, Expiration, Code) are entered, the details are sent to a Payment gateway. This payment gateway acts as an interface between the website and Merchant account bank. This payment gateway ensures that there is a secure way to get A's details over to the bank.
Then this payment gateway leads to a payment processor.
This payment processor is responsible for routing the payment request to a merchant account that will accept the payment on behalf of buycheapkeyboards. Since security of payments is complex and is potentially something that can put a business out of it, we need to decouple where the websites end and the banks begin. The merchant account is an account that each business (like buycheapkeyboards) must have to accept payments.
Lets take a breath now, a few more terms are going to be throw at you.
Now the merchant account needs to check with the issuing account. The payment processor ensures that this 2-way communication happens. The issuing account is the bank account that A has. These 2 talk to each other and get approval for the payment. Then, if the funds are enough in A's account i.e. the issuing bank account has enough funds to transfer the payment, then the merchant account gets the money. The merchant account is also called a merchant acquirer. The issuing bank is also called a card issuer.
In a nutshell, thats about it.
Now there are issues with fraud and chargeback. For chargeback, consult this helpful info-graphic from
Chargify
Credits:
http://www.quora.com/How-does-credit-card-processing-work
http://www.quora.com/Online-and-Mobile-Payments/What-are-some-innovations-introduced-by-recent-startups-in-online-mobile-payments
http://www.quora.com/Online-and-Mobile-Payment-Startups
http://allthingsd.com/20130117/is-there-a-big-win-for-the-payment-startups/
http://shopsite.com/cc_101_detail.html [Just replace Shopsite with your own payment startup]
http://www.wufoo.com/images/powered/paymentform.png
http://chargify.com/images/uploads/chargebacks-v3.jpg
Of Yelp and Foursquare
·
I have not been a big fan of foursquare. I think Yelp also has its issues. But when I am hungry and in a place where either 1. I am sick of the regular place or 2. Lost; then I turn to Yelp. It hasnt disappointed so far and the "Open Now" filter has been a blessing. Foursquare wants to muscle in on this territory. Fair enough, so do a lot of companies and it is a lucrative market. After all it does not get any more contextual than where you physically are present at this very moment. So what are we looking for and where are the main issues that we want to tackle?
1. Obsolescence - Miyaki was a frightfully popular place in downtown Palo Alto, it closed. Yelp did not know about it immediately and had some lead time before this information percolated through its systems. When we are making a catalog, data goes stale and keeping that data fresh becomes a dirty job with no proven approach.
Yelp is still very much a catalog. Foursquare on the other hand uses check-ins, actual people checking in to a place. If the place ceased to exist people would stop checking in, the firehose would dry up.
2. Answers - An aggregate review is a very poor measure of a place. I went to the Grand last weekend and wrote a review which was buried somewhere so deep that even I cant find it. I want to do X on a Saturday night with Y number of friends. What are the chances that I have a good time? You would have to sift through a ton of rubbish to get what you would want. Somehow search does not seem to be the right answer here, it needs analytics in addition to search and sort. A social conversation is more likely to answer a question than greet you with a wall of text.
3. Engagement - So you read a review on Yelp and go to that place which has a 4 star review. Or you are deciding between two equally cool nightclubs. You know what would sweeten the deal? A discount, a deal, pre-booking - absolutely anything that you can throw at an interested customer is better than not knowing what they are doing with your information. Yelp has a check-in system which I am not sure how many people use but Foursquare was built from the ground up for people going into places.
So, hand the fight to Foursquare and bet the future on them? Not yet. The network needs more active users in more diverse places to gain traction and actually be useful. It needs to be the de facto standard for reviews and it needs to balance the deals going out and whoring out your customers. Its delicate, its exciting and over the course of the next year, there is no other company better equipped to upstage Yelp.
1. Obsolescence - Miyaki was a frightfully popular place in downtown Palo Alto, it closed. Yelp did not know about it immediately and had some lead time before this information percolated through its systems. When we are making a catalog, data goes stale and keeping that data fresh becomes a dirty job with no proven approach.
Yelp is still very much a catalog. Foursquare on the other hand uses check-ins, actual people checking in to a place. If the place ceased to exist people would stop checking in, the firehose would dry up.
2. Answers - An aggregate review is a very poor measure of a place. I went to the Grand last weekend and wrote a review which was buried somewhere so deep that even I cant find it. I want to do X on a Saturday night with Y number of friends. What are the chances that I have a good time? You would have to sift through a ton of rubbish to get what you would want. Somehow search does not seem to be the right answer here, it needs analytics in addition to search and sort. A social conversation is more likely to answer a question than greet you with a wall of text.
3. Engagement - So you read a review on Yelp and go to that place which has a 4 star review. Or you are deciding between two equally cool nightclubs. You know what would sweeten the deal? A discount, a deal, pre-booking - absolutely anything that you can throw at an interested customer is better than not knowing what they are doing with your information. Yelp has a check-in system which I am not sure how many people use but Foursquare was built from the ground up for people going into places.
So, hand the fight to Foursquare and bet the future on them? Not yet. The network needs more active users in more diverse places to gain traction and actually be useful. It needs to be the de facto standard for reviews and it needs to balance the deals going out and whoring out your customers. Its delicate, its exciting and over the course of the next year, there is no other company better equipped to upstage Yelp.
Sharing pictures and making scriptures
·
This post on TC was pretty descriptive about the murky waters of photo sharing startups. Instagram and its monster exit has led to a plethora of startups doing Instagram + some additional features. There is a lot of movement in and around this space. Conceptually, photos carry so much traction because
1. Density of information - A photo is not just a way to capture a moment, it is also an instrument to preserve your memories, to think back and reflect about what you were doing when that memory was formed. It is also an effective way to signal. How will the world know that you are having fun if you dont post pictures on FB. A rough survey about the golden rule of the internet on reddit revealed that the 1st rule of the internet is indeed "Pics or it did not happen".
2. Simplicity of capturing this information - Even with your phone unlocked, you could snap a picture. Use a filter and you are your own agency. It is simple and very easy to use. Pretty much anyone can pick it up.
3. Means of expression - Photos lie at the intersection of expression and inspection. You want to have moments that are intensely private - like your family's pictures as well as photos that are for the eyes of the world (Think of the firehose on your newsfeed that Coachella accompanies)
4. Potential of this expression - Filters can do in an instant what the promised land of Photoshop used to be. You can do with your unused pictures what Sci-Fi movie makers in the 90s could only dream of.
Regarding the density of information,
There are the sticklers for the quality of photographs who wont touch FB photos. This is a niche group that does not want to see their precious quality get diminished by some hosting service, never mind if this service was 1 Billion strong. As Amazon EC2 grows and the costs diminish more, I would expect convergence on FB. The portfolio sites should not have a strong enough value proposition to continue. Or portfolio sites need to go the way of Medium and attract class content on the sites.
About the simplicity of capturing this information,
Can clicking a photo get any easier? I would wager not.
The means of expression is where this gets interesting,
Imagine the world having to know that you are having fun - classic Facebook. Imagine pics of your family and the closest friends that you have ever had, the ones who know everything about you from your internships to the interest you earn on your savings account. Think theres a potential problem to be solved? Path beat you to it. Is there potential to improve? I personally have a path account and like everybody, I love their design. I do not use Path much but that is probably because I do not have many friends I want to open my heart to.
Now over to the potential of this expression,
Thinking beyond filters, this is where the real potential seems to lie. Consider a few use cases
1. Family Vacation - I have an iPhone that auto uploads photos to Dropbox so I never have to manually take a backup again. There are 4 iPhones shared within the family. A grand total of about 250 photos and you have no idea which ones are worth keeping, discarding or sharing. You take all the photos and ask that they be added to a common sandbox. Here, an app iterates through the photos, face-matches your pics and sends the ones that you are in for you to categorize. This happens for all the photos and each member of the family is uniquely well positioned to identify good pics with them in it. Where this breaks down is when there are group pics or nature shots. A general voting on those photos and they are packaged for you to share in any way that you like.
2. Time travel - Those kids sure do grow up fast dont they? With this time travel app, you could keep time shifted records of things that grow rapidly. Pet owners form a lovely target user group. So do parents.
3. Group Pics - "We can just photoshop her in". Imagine an app that has a singular point of focus, to make a meaningful group pic.
And of course, then we move on to the holiest grail of them all.
The merging of the conceptual world and the real world. Imagine this great park that you have visited. You want to know if the hike up to the reservoir is worth it, or if the taco stand is just expensive or really good, or if the make out point a half mile away is worth a quick sprint after sunset. How about what did Jack do if he had 6 hours to kill in SF? Yelp and Foursquare are doing this with restaurants and cafes. The potential to enhance this body of knowledge to be malleable to the conceptual framework of reality is almost limitless. If anyones asking, Foursquare has a better shot at doing this than Yelp.
And thats where the next round will begin. Fight!
2. Time travel - Those kids sure do grow up fast dont they? With this time travel app, you could keep time shifted records of things that grow rapidly. Pet owners form a lovely target user group. So do parents.
3. Group Pics - "We can just photoshop her in". Imagine an app that has a singular point of focus, to make a meaningful group pic.
And of course, then we move on to the holiest grail of them all.
The merging of the conceptual world and the real world. Imagine this great park that you have visited. You want to know if the hike up to the reservoir is worth it, or if the taco stand is just expensive or really good, or if the make out point a half mile away is worth a quick sprint after sunset. How about what did Jack do if he had 6 hours to kill in SF? Yelp and Foursquare are doing this with restaurants and cafes. The potential to enhance this body of knowledge to be malleable to the conceptual framework of reality is almost limitless. If anyones asking, Foursquare has a better shot at doing this than Yelp.
And thats where the next round will begin. Fight!
Big Bag of Data
·
So data < information < insights
Having known this, why would we think that more data would help bridge the gap between data and information?
That is in essence the fallacy of big data. Big data will expand the range of hypotheses, it will also extend the range of the search space. Nothing gets easier with the addition of more data, especially when you are in the business of data exploration. The main job of a data scientist is to find possible hypotheses that describes what is happening with the data. The data does not lie, or at least lies less than the human :)
Should we collect more data? Should we care?
Just because data has grown exponentially, does not mean we keep collecting it. We care if the data is interesting. If the data fits snugly into an equation, we have figured out the data. There is no mystery left with the data anymore then we should not care about it. It is with figuring out the various "Whys" of the data that we can find true insights. We should only care when the data puzzles us.
Where do insights come from?
Not from no data. That double negative is just meant to express the fact that there is no accepted answer. Data helps and some data is better than no data. But where we should draw the line is highly debatable. The storage costs per gigabyte reducing by a factor of 6 has meant that we can safely store information without having to worry about cost overhead but it is in the utility of this data that we are confused. Big, huge quantities of data are not necessarily high yield in terms of insight. And it is here that having a lean approach towards storage can certainly help.
In conclusion
Store the data to figure it out. Once it fits, its time to move on to another data set. It is in these attitudes of exploration that we can truly drive insights or at least a mindset that helps achieve insights.
Riffed off of and thought credits to
http://techcrunch.com/2012/11/25/the-big-data-fallacy-data-%E2%89%A0-information-%E2%89%A0-insights/
Having known this, why would we think that more data would help bridge the gap between data and information?
That is in essence the fallacy of big data. Big data will expand the range of hypotheses, it will also extend the range of the search space. Nothing gets easier with the addition of more data, especially when you are in the business of data exploration. The main job of a data scientist is to find possible hypotheses that describes what is happening with the data. The data does not lie, or at least lies less than the human :)
Should we collect more data? Should we care?
Just because data has grown exponentially, does not mean we keep collecting it. We care if the data is interesting. If the data fits snugly into an equation, we have figured out the data. There is no mystery left with the data anymore then we should not care about it. It is with figuring out the various "Whys" of the data that we can find true insights. We should only care when the data puzzles us.
Where do insights come from?
Not from no data. That double negative is just meant to express the fact that there is no accepted answer. Data helps and some data is better than no data. But where we should draw the line is highly debatable. The storage costs per gigabyte reducing by a factor of 6 has meant that we can safely store information without having to worry about cost overhead but it is in the utility of this data that we are confused. Big, huge quantities of data are not necessarily high yield in terms of insight. And it is here that having a lean approach towards storage can certainly help.
In conclusion
Store the data to figure it out. Once it fits, its time to move on to another data set. It is in these attitudes of exploration that we can truly drive insights or at least a mindset that helps achieve insights.
Riffed off of and thought credits to
http://techcrunch.com/2012/11/25/the-big-data-fallacy-data-%E2%89%A0-information-%E2%89%A0-insights/
Subscribe to:
Posts (Atom)
