Domain specific BigData companies to watch !

1. Automotive Domain - Dataium
2. Credit/Finacial - BigData Scoring  /  ZestFinance
3. Food Industry - FoodGenius
4. Marketing / Advertising  - BlueKai  /  Metamarkets
5. Retail - RetailNext
6. Healthcare - Apixio
7. Machine Learning - SumoLogic
8. Sales / Marketing - Lattice-Engines
9. Real time Pricing - QuickLizard

Data Scientist - 5 Things you can do to claim the title !

1. Know - What is Data Science?
The best definition I found for data science or data scientist is by Josh Wills, Director of Data Science at Cloudera:
"Person who is better at statistics than any software engineer and better at software engineering than any statistician"
You should view this presentation by Carlos Somohano founder of Data Science London to understand the big picture:

Big Data [sorry] & Data Science: What Does a Data Scientist Do? from Data Science London

2. Learn - R (basics and beyond)
If you did not ignore step#1 and absorbed the slides and links above then you probably know what R can do for you . The easiest way to learn R is to start programming with R. I recommend installing R studio on your laptop. This IDE comes with R help, console, workspace and editor all in one. Before you install R studio, install R first from this link.
If you are organized person, you start with official R tutorial.
If you are impatient and want to be able to use some built in packages to produce some nice plots, you follow this tutorial.
3. Learn - Machine Learning - Join Upcoming Courses on Coursera
If you have willingness to learn from the masters - you are in luck ! Coursera is starting free course on Data Science on May 1st.
You can find other related free courses on Statistics and Machine Learning !
4. Learn Python - yes, you will be glad you did !
Again, my favorite place - Coursera. Check the course - An Introduction to Interactive Programming in Python. It is starting on April 15 - see its your lucky day !

5. Last but not least - show some of your good work to world and claim your title !
Create some cool analytic project, show off the model you have developed - make sure put it on github
link the project to your linkedin profile and claim the title you have been craving for !
Also try to validate your "Data Scientist" position by participating at Kaggle - an online community of Data Scientists.

Project Ideas III

Here are some more project ideas for Students:

1. Small business advertisement channel - What is the best and most profitable advertising medium for a mom and pop business like coffee & donuts shop, a neighborhood dry cleaner, or a sushi restaurant?

small businesses have least amount of money for advertising so it has to be extremely effective. The app needs to figure out based on location, demography, business domain  and sales data, which is the most optimized channel for advertising? is it local news paper ad, google ads, facebook, penny saver or something else?

This project requires building statistical model /or use R with social data analysis 

2. An Android / iPhone app for matching household handyman jobs to contractors. - People need help post the job. contractors bid on them, customer chooses the right bidder (may not be lowest) you need to have a star rating system based on their past work and reviews (elance like system)

Job seekers can set and get alert when a job is posted in specific category, so they can bid from anywhere..

payment system is built in so when job gets done money is paid and platform gets a % cut in every deal.

3. Home invasion and Robbery data visualization - UCR http://bjs.ojp.usdoj.gov/ucrdata/ publishes data about property crime. can you find city / county wise data and create cool visualization tool to highlight these types of property crimes. You can also relate this to property value in the market to show correlation. 

4. Virtual Farmer's market - connect Healthy eaters with local farmers  on virtual farmer's market. People  order online and pick once or twice a week in designated neighborhood area or delivered fresh from farm with extra shipping either to your home / work. Benefit ? - guaranteed to local and fresh. This trend is also picking up in trendy technology startups. They tend to serve their employees organic fruits and other stuffs free at workplace.

5. Web based analytic tool framework - web framework where data scientists can drop their model(written in R or weka) , upload data and generate model output and visualize them. They can save the visualizations, and model output as well for further processing. 

More later...

MS Software Engineering Project Ideas II

Its time again  for students to find innovative projects. I am bombarded with emails from my students on this subject so here I am posting some of my ideas again. Feel free to discuss any with me.
I also posted few in my last Post on the subject. Some of them may have been chosen by my past students.


Project#1 - Measure productivity of your employees
It has been found that within few years:

• workers will collaborate 80% of their time
• work will flow seamlessly, fluidly, across fully connected and visible processes
• agility will become key performance metrics
• workers will evolve to be more client focused
• pro-active capabilities will be strengthend with predictive capabilities
• work will not be centered on an organization but on a community of people 
• batch processing will be forgotten, real time will be the norm

Create a tool/portal for employers to use various mathematical models to prduce some of the metrics discussed aove, so that employer can maximize productivity and employees can maximize skills and professional growth

Project#2 - Intra-networking tool

A tool/portal to allow your employees to network by:

•  pairing up for lunch  -    employees (executive or regular) sign up and system pairs them up for lunch together either at company cafeteria or outside restaurant (promotes openness and collaboration)
• pairing up for upcoming project - employees sign up with their interest and availability and system uses, technical skills as well as their social interaction data to pair them or group them for upcoming project
• pairing up for mentee/mentoring  

Project#3 - WomenRule
Gather data on executive leadership team of fortune 100 companies and visualize the gender participation

• same with fortune 500 companies
• do the same for other countries 
• come up on some cool analytics... which sector has highest participation of women and men etc...
• even go further by providing features and plug-ins for personality development, leadership training etc..

My own 10 tech predictions for 2013

1. "BigData" buzz will reach its pinnacle in 2013. Predictive analytics will become part of all applications, small or big. Small biz customer centric app will embrace these technology for their competitive advantages in big way. For IT professionals like us acquiring statistical and predictive modelling skills will be the key.
2. Low power computing will emerge winner. The unprecedented success of tiny Raspberry pi (close to 1 million units sold)  is teaching kids all over the world fundamentals of computer sciences. Everybody should take a bite. I took a bite too. This was my Christmas treat to myself. ARM based computing is already giving powerhouse Intel run for their money. 
3. Free online professional classes (Udacity/Coursera etc..) will go Freemium model. They might start offering a group of related courses as part of certificate programs and might tie up with local community colleges for classroom engagement to make them more effective. In any case, this model is going to get bigger and by 2015 if not now will give big brand schools run for their money. 
4. Mac will become platform of choice for independent software developers. Employees will be encouraged for BYOD (Bring your own device) Mac to work and they will reimburse the employee as well for the device up to certain limit. Some startups already do this.
5. Traditional databases in cloud will not fly. NoSQL databases will be preferred choice for WebScale  database. Some NoSQL databases will be trying to accommodate ACID properties. Others will stick to new meaning of ACID (Associative,  Commutative, Idempotent and Distributive) 
6. Data warehouses will move towards appliances model. More plug-and-play refrigerator like systems will entry into market. These appliances will have easy load of data, analysis and dashboarding built-in.
7. Micro lending platform will make ways into many other territories such as Student loan. Some regulations are already being approved for such loans. 
8. Every organization will use social software to uncover hidden (under water) organization. For example , employees emails and chat discussions will be analyzed by the machine to come up on analytics that can be used to improve collaboration and efficiency.
9. Data Scientists will be one of hottest job. This will prefer PhD in Mathematics/statistics but comp sci professionals with analytic experience will be preferred.
10. Javascript skills both at client and server side will be high in demand. Those with mobile software development experience will have plenty of opportunity both for salaried and freelance jobs.