Data smells too

Have you heard of the term “Data smells”. As developer, I heard about “Code smells”. In programming, Code smells describe a combination of signals that indicate larger, deeper problems. I started to hear “Data smells” in recent past and similar to code, analytics can stink, requires a considerable effort to solve the problem.

When one looks in to data for insightful answers, one ends up uncovering more quality problems?. Hence with little bit of scrutiny, results change drastically. Different parts of the organizations have different version of the truth about same data. Here are some reasons why smelly data gets in to the system at the first place.

In organizations, line managers, with little or no training in data science are the consumers of data analytics. They are not able to judge the quality of the research — or determine whether a project should take as long as it does. Less experienced data scientists sometimes ignore the experience/insights that line managers could offer. These insights can improve the result or shorten the research process.

Analytics compiles data snapshots that happens in the world and these may not fit into well-structured or clean models. In addition,the world continues to change, even if the systems don’t. A system made sense in isolation or at the time it was developed. How to make sense of generated results connecting disparate sources of data?

Organizations are looking to obtain insights in shorter time, leading to short-term solutions. Once short-term solutions are in place, it becomes tricky to find time to go back and create longer-term practices that are more robust. Do leaders have required discipline to decide where to standardize now versus later? Can organization have capability to invest resources upfront to ease later growth?

In programming, refactoring is performed as series of small changes to change the internal working of a system without changing its observable behavior. This has inherent risk in the short term it can break working systems and absorb resources without apparent payback. Removing stench in data is easier said than done.

  • Impact or payback from investment done in analytical refactoring is not immediate. Cost are spend immediately. We are back to Now versus Later.
  • Identify the areas that need attention. Find approach to perform refactoring activities decoupled from major projects. Effectively reduce risks associated with the larger, more strategic projects.
  • Like incremental/iterative approach to software code, an incremental strategy applies to refactoring analytics. For example, start with “what is the data inconsistency do the most people struggle with?” An incremental approach can reduce risk & help people to see benefits earlier.

If you think there would be no time to clean up data, make sure that you are not spending time to mess up. Be aware that it may be far easier to avoid creating problems than it will be to clean them up later.

Me@Innofest2015

Let me start with apprecation for every innovator who dream and work to make their dream to become true. This includes innovators who could not come to Innofest. It was refereshing to see a lot of college students presenting their innovations. Some students have started to cultivate passion across their education journey and are working to continue to nurture their dreams, being at job in different industry. For change, I felt that innovators focused more on their innovations, rather than monetization. Makes me think what could I have performed different if I was born 15 years later.

Started first visiting TeamIndus. The team had done good initiative in science. I expect them to look beyond Google Lunar X prize and have people mission also. While their innovation gets used in real world, I visualize that innovation can serve as motivate college and school students to dream and take action to realize their dream. What is plan for TeamIndus to engage with students?  I see them more engaged with space agencies, corp-orates, old men. Not clear with schools/colleges.

Started with session on “How do you create a condusive environment in your office to build a culture of innovation?”. Mohit shared about removing unnecessary HR process and bringing focus on faith and belief on employees. He encourage employees to innovate was”Think through the day and write down 3 things that bothered you. If the same bothers you on  regular basis, write a plan to solve and discuss”

Failed to understand Alok speak around Focus. Some good points for sure. Focus is important in startup journey. I liked his question to startups “What you know about business that no one else know?“.  Touched upon the decision to chase one rabbit or multiple rabbits(ideas) in a startup and challenges. Shared Google laser focus to develop tools/system around BigData, rather than search and ads.

Heard Ganesh(first time) explain  business models & their impact on the money and speed needed to sustain and grow business. Contrasted business with need of continuous sustained capital to survive for long years(eg:Flipkart) and business with no need for continuous capital raising and depends on customer pay(eg:School). Contrasted business with demands to grow fast (eg:Taxi or Food app) where one needs to get to place 1 and 2 and business allowing space for more folks (schools/hospital/restaurant) without nature of  “Win, Take All”.

I have posted my impression on first hand about innovations in blog post Innovations@Innofest2015. Thanks to volunteer team and IISC for making this event happen. I see Innofest2015 is a great start and wish following.

1. Make Innofest festival to happen in multiple cities
2. Open Innofest stalls for schools and college students to visit
3. Bring more of innovations that are connected with fields other than IT based.
4. Get active participation of academicians who are real innovators. Take inclusiveness to next level.

Felt that speakers of concluding event talk more about innovation than investing  in innovation. As I spend most of my time with stall, I attended few sessions. The sessions were more of success and not sure representation of failure and resilience as  inherent part of innovations.

One area to improve that I noticed was to reduce the plastic waste created by food stalls. Both vendors providing food for participants used plastic. Interesting to note that they could serve simple food without using package/plastic material. Happy to see water cans with paper glasses and no water bottles.

Innovations@Innofest2015

Here is small write-ups of stalls I visited and could relate better with innovations at stalls@Innofest. Wish that innovation stalls were open for viewing by public,children and youngsters. Captured innovations to share and motivate readers to dream.

Let me start with TeamIndus,prototype design and aerial vehicle. Young guys have been able to innovated on large scale and serve as motivation for all to dream big and achieve.They stand to demonstrate that fundamental innovation can happen in India too.

Kisan Raja , powerAdapter to enable farmers to swtich on motor pumps in the field from his home and save the effort to walk across a long distance.

Visted stalls of 3D printers PRAMAAN and PRAMAAN-Mini. Learnt  what items can be made with 3D printers and raw materials used to create items like keychain, table tops, flower pots. Wish the innovators work to enable audience to relate to product easily, something like create some pre-designed items, make items and share with audience to take home(for charge). Connected audience has ability to share with more people.

vSkin is a wearable glove to creates a sense of touch on the users’ fingers in a virtual setting.  One can feel the hand movements while working with a digital piano or use glove as part of video conferencing calls to perform remote handshake. The innovators are currently in age group of 19 to 22 years.

Tesseract is a Virtual Reality headset. The prototype headset works with content created or stored on a laptop; new ones work with mobile phones also. Wish Tesseract becomes complimentary item with XBox for customers to think about. They have sourced optics from China, acrylic lens from India and manufactured in China.

RideLogik helps with charging base for the smartphone. Wish they provide more value propositions around the vehicle efficiency and performance. Though they can integrate with any vehicle that runs based on electronic control unit, their focus is 2-Wheeler’s. They need to develop channels to reach existing 2-wheeler owners.

Found innovation of insulin pen to enable diabetes patients to take care of their health better more connected at ground level.The innovator has patent filed In India.
.

Shrishti  has displayed multiple innovative items. Though I am not covering all of them, I would like to cover the chair they have made using old newspaper. This chair can be used for outdoor purpose too and would not spoil in rain.  A means for paperwallas to become furniture makers? Here is my photo on the chair to say that chair was stable with my body weight.
.

Plugzee is device to turn any audio speaker system into a Bluetooth speaker. The device plugs into the 3.5 mm audio jack of speaker & lets you stream music wireless by Bluetooth pairing with a smartphone. They have leveraged crowd-sourcing to transform innovation to reality. Already started to ship last week- that is nice.

For more innovative items present in the stall, please check Madan article Passion and policy – startups and ministers at InnoFest 2015!

Edited version of this article was posted at http://pn.ispirt.in/innovation-innofest-2015/

Did you get Freedom on 68th Independence day?

Being lazy, I did not post a  blog in last few months. Last weekend was independence day. and I had some interesting observations and shared them here.

My children had independence day function at school. My apartment has flag hoisting followed by activities performed by children in the morning and we moved to breakfast. Observed that persons from catering service served serving food, compared to earlier practice of residents volunteers. The breakfast attracts people to attend independence day event.

Observed that people did not check whether food is available for other residents and were more interested for re-fill. Some mothers asked permission to take an extra plate for children who went to school. One can see this as freedom earned by mothers & housewives from activity to prepare  breakfast.  We also made use of paper plates and  paper glasses compared to earlier polluted practice of plastic/thermocol plates for food. Few waited to ensure that support staff had food too, support staff work more on event days. Can support staff enjoy freedom at same time when others enjoy freedom?

My proposal is to keep hundiyal or charity box. Ask residents to contribute money,equivalent to cost of breakfast and pass on the collected money to charity. On sharing with residents, the idea was welcome with suggestions for me to take lead to implement same during next event.

As children had school in morning, talent/cultural show was planned in evening. There were a lot of good activities including a “New Era Ramayana” skit. My daughters acted as Seeta(big) and Supernaka(small) in play. Jayant, resident & father did amazing job of creating skit with children. In addition skit was enjoyable. He was inclusive in skit. Awesome.

There were food and game stalls setup by adults and young children,(future entrepreneurs).  I had setup a charity stall with aim to collect money along with entertainment sped and create awareness of getting happiness by sharing. Observed  external girl(10 years) had come to sell balloons. On enquiring whether she wants to view event and not sell balloons, she wanted to sell as her family needs money & here father was selling too.

I was feeling both happy and sad at the same time. She sold 75 balloons and she did not enjoy like other children. Later, balloon girl’s father came. To reduce my guilt, I asked my daughters to go along with the balloon girl and buy her needs from stalls. She came and thanked me. Felt quite small. When does she get freedom?

Interesting events at end of event. Two boys who ran a stall came and donated Rs 15. Two girls donated Rs 50 and Rs 100 for charity, out of money provided for stall purchases. One adult stall owner came and donated profits of Rs 700. Felt good and satisfied with awareness created by my stall. Though children donation was small, their contribution was more in terms of compassion and consider more than my charity stall collection of  more than Rs 6000.

Next day morning, we had sports event and children and adults had great fun. Even grand parents had activities designed for them. Still observed that support staff was working tirelessly to make event success.  I also saw them working to put chairs and places back in its place across sports day. When support staff gets their freedom?

To encourage children to be more caring and to appreciate their generosity, I shared with them some t-shirts designed by children in corporate annual picnic.

Are we aware of change in our buying behavior?

As I read  blog What is a sale for? (48 hours) , I got reminded of my childhood where my father shared economics and commerce principles

1. Categorize expenses/items as necessary and luxury. The necessary included simple food items, simple clothes to wear, bicycle, books, shelter to live, education and health and luxury included television, tape-recorder, motor-cycle computer,electricity and motor-vehicles.

I remember the categorization of luxury and necessary & observe that luxury items of the past like backup generator, phone and micro-oven are necessary items of today.

2. Choose purchases based on “Value for money” approach. My father used to take me to santhai (market place) to purchase fruits and vegetables. He always purchased good quanity of fruits and vegetables that are in-season, available in large quanity at low price. To explain better, avoid off-seaon fruits and vegetables, available in small quantity at high price. This enabled us to eat all fruits and vegetables.

I use this approach in different perspective to book vacation at hill stations. Before 31 March, it is off- season and accomodation comes cheap, compared to rates on 1st April. No significant difference between last week of March & first week of April. .

3. Comfort and Convenience come at extra cost. In Santhai, he had made me observe that same items priced at high price in shops closer to the entrance was priced at low price when shop was deep inside the santhai. While I cribbed dis-comfort to walk a long distance, he explained cost addition for comfort need, adding “Once you are comfortable, it is not easy to go the other way. Keep away from comforts”.

Today I see that in name of comfort and convenience, new economy are created in terms of air-condition, app based cars, e-comemrce and so on.

4. Prepare a list before going to shop. Visit shop with a items list to purchase. Stick to your list. Avoid purchase of items not planned in the list. As money is present, do not perform instant purchases. Estimate money to buy items & carry little more than required cash. A practical way to control urge to make instant purchase and be aware of fluctuation of prices at shops.

Today I am cost and quality conscious. On windows shopping, I share to the shopping agent of my plans not to buy and recommend him to spend time with other custtomers. My kids presence does make me perform instant purchases.

5. More liquidity (cash in hand or savings account) leads to more spending. Track previous month expense, keep delta as emergency & save remaining as Fixed Deposit. This also protects you from instant purchased and buying without plan. Whenever we need money, we estimated the need and withdraw cash from bank.

Today I am aware of more banking functions than young people working in bank. There are no physical salary checks & salary is credited to our banks. Few visit banks due to presence of debit cards that allow access to entire money in the bank account. There are credit cards to allow access to money that comes from future earnings.

Have I “Moved on with life” from mother’s loss?

On mother’s day,I started to think about  a missing piece in my life. My mother died when I was 12. I lived with her between age of 8 to 12 and memories are fading with passage of 27 years. On her death, I did not realize what it means to be without her. I have heard that I did not cry on her death. Till I was in school. I used to talk of the neat green envrionment of burial ground.

The older I  grow, I remember and also realize the absence of my mother. May I have not bereaved for her death makes me less capable to handle emotions when I feel down.  I am still carrying the bereavement rooted in childhood, which has left emotional scars. I feel that my life would be “much better” if my mother was for more years.

When I look back, few people including educators can recognize signs of loneliness, isolation and depression in grieving children. Children fear that they would break from other loved ones too and want to experience that there is some continuity in their lives. and that they are cared about. They look at relationships in terms of what people do for and with them and whether they are made to feel safe in their world.

Support groups that help grieving adults are less effective in terms of children as children need attention and people who encourage them to talk about their feelings. When you are adult, you have the benefit of being with people who have been been through it. When my mother died, I didn’t know anyone who’d lost a parent and that made me feel alone. I could never relate with adult who shared losing his mother.

On becoming adults, their wife & children might underestimate phenomena of losing parent in childhood. Not easy to empathize when one did not experience the chronic pain. The person is better understood by persons who lost parent in childhood.

From 12 years, I started to fell that I was not a kid anymore and need to be more resilient, responsible and independent. This was result of work performed by me in my school years in diverse areas of banking, equity shares, finance portfolio and non-profit work. My wife reports a childish behavior at times and I am dependent and not responsible at home. Do I long that some one takes care of me like a kid? – a contradiction. Have I  “Moved on with life”?

During entepreneurship, a friend spend mostly for his family and did not spend on himself and  His mother purchased dress for him during festivals. When his mother know of his worried of failing to pay salary to employees, she offered to break her fixed deposit to see whether pain can be dissolved. This incident made me feel absence of mother who puts trust and faith, even when things go wrong for her child. Also unfair to expect unconditional motherly support from wife with children. Have I  “Moved on with life”?.

My mother’s sister got jobs and moved on with their life and I lived with my my father. My father worked hard and travelled. I came to empty home for a year with a limited support of few days visit by my grandpa. It was lonely and abandoned. Today I continue to come home late and sit in living hall eat alone my food and rarely realize that there is no family meal at home. a contradiction. Why continue to feel lonely and abandoned with wife and 2 children?. Have I  “Moved on with life”?

Experienced eating at uncle’ place, giving money. Quickly realized of being a burden to them. confirmed not getting quality food. This lead to me having food in hotels 3 times a day and mostly alone. Not sure of next meal, I ate more(like camel). Though life had changed. I visit hotel/restaurants with family, introducing them. Though I am fine to eat outside, I long to eat home food. a contradiction. Have I  “Moved on with life”?

At times, my father would cook and that means more work for me, perform cleaning utensils, cutting vegetables and so on. a contradiction. Today I keep myself away from kitchen and look for food to be served and do not offer suggestion of what to prepare or help in kitchen. Why I have become like this? Have I  “Moved on with life”?

My father believed that he can take care of the boy. He had dreamt of living a life similar to Rama(with one wife) and was more grieving his wife’s death. In school days, I experienced more a proffessional touch due to my father and there was no experience of personal touch.  I treat all people I know as friends and expect personal touch on default until I realize that it is wong place. I consider them still as friends and support them in their needs the best possible and invest time in building relationships. Not knowing how to separate my public, business and private life is a challenge for me.  Have I  “Moved on with life”?

Losing a parent at a young age make the child feel helpless and even ‘different’ from the others…there is a lot going on in the kids mind and these things have a major effect on them. They give raise to contradicting straits that makes them more emotional

  • Reallize of becoming more emotional undergoing major life transitions or witness a major life transitions on television. At the same time, you can relate more with other’s transition like getting married, having a baby, death and separation and offer help.
  • You want to get close to person and become close to person and then suddenly avoid being close to minimize being hurt again by another loss. You need space and time to become close and on being close, you do not disengage soon.
  • You fear loss and avoids everyday situations that trigger feelings of loss. Though you have ability to be more empathetic, your fear leads anger and emotions, with sense of loss in relationships. While you are more hesitant to form relationships, you handle relationships safe, aware that you actions can lead to losing relationships.

Having written this and being adult today, I wish that I can move on with life keeping aside my mother loss after grievance.

EAST INDIA COMPANY IN TODAY’S CONTEXT – PART 3

When we move to start of IT landscape in late 80s and 90s, India had a lot of hardware companies. Wipro and HCL Tech started as hardware companies in 1990s and were struggling to create PCs and printers. The government did not provide any special support to these companies.

Here comes globalization. The foreign companies called MNCs arrived to sell their PCs and printers. MNCs partnered with Indian companies to assemble products locally. Assembling  MNC products was more profitable to Indian companies rather than doing R&D and also manufacture of their own products. MNC products provided more money on short term and this was earned with less effort.

MNC hardware products took over the market. To reduce transportation cost, they manufactured hardware in South East Asian countries and shipped products to India. This caused competition for remaining Indian manufacturers and they started gradually moving away from creating their own hardware.

At this time, a new opportunity called developing software promising more money with less labour problems, less man power and less infrastructure and the Indian companies moving to software. Now all hardware market was left open for MNCs with little or no competition. The companies developing software also became customers for MNC hardware.

Indians with expertise in English Mathematics, (trained by British, as explained earlier) qualified as good clerks were suited to develop software/write code and mainly in repetitive tasks. To handle the challenge for MNCs in communication, some Indians st travelled abroad for short term assignments and Indians started to migrate for new opportunities to USA.

India has state owned telephones and communication channels was primitive. India created CDOT to start telecom revolution, leading to more phone usage in India and enabled doing business simpler across cities. The upgrades in telecom industry enabled foreign MNCs to sell more of their hardware products to telecom industry and existing business.

Like rail road transport, came the advent of internet, leveraging the telephone line. Indians with less skills of English and Mathematics were good and cheaper to perform repetitive tasks using phone and internet. Solving communication challenge gave rise to BPO industry. Again glorified clerks who earned better salaries took risk to start BPO outfits to employ clerks in large numbers. Once the company had good number of clerks, the company was sold to foreign MNCs on validation based on head count.

Indian software industry earned money to create products and solutions for MNCs. They got 100% tax exemption for export oriented software. Doing software for abroad was more lucrative than developing software for India. Some of the founders made a lot of money and became wealthy. Few companies came in to market that created products like Tally and faced a tough environment to acquire talent.  Why would Indians join Indian product company with uncertainity, rather than joining Indian software service companies, travel abroad  and make more money?

Whether products or services, companies from both order enabled purchase of PCS, phones and laptops. Clients of the software products had to purchase hardware to install software. These hardware products were primarily manufactured by foreign MNCs and helped them to earn good money from Indians. On one side, MNCs  got more software work done by Indians at lower cost compared to their country. On other hand, Indian purchased their products at higher price and made their business strong.

Thanks to English knowledge of Indians who were computer savvy, MNCs sold hardware products with only support in English. No one asked for support for Indian languages and Indians learnt spending monet to use software available only in English. Be aware, MNCs supported all European languages for hardware and software sold in European economies and had no pressure to provide similar offering to Indians, though population of some Indian states is larger than European countries.

Some Indians started to build websites and web applications for Indians and computerize the Indian government process and local processes. MNCs entered government arena and offered software close to free cost  as freebie when organization purchased their hardware. This provided a tough competition to Indians software companies and some of them decided to develop software only for the MNCs.

The software came along the hardware was also available only in English. See a divide and conquer strategy working underneath the economy. To use software, Indians with knowledge of good English and programming and became software clerks or BPO clerks. I call them software clerks (also called software engineers). These guys were trained to be expert in specific area and not to develop the entire product or solution.

The design of product or solution was done by MNCs in their own country and the execution of repetitive steps was done in India. This looks exactly like their ancestors in 1800a and 1900s. By paying these software clerks more salary, they created a habit that is difficult to change or revolt against when you realize its pitfalls and want to have your own startup.

See till now, they leverage the English and maths talent in Indians. The phone lines did not provide them the communication infrastructure to scale. Here comes the internet that introduced new business jargons of websites and web applications. These could be deployed at one place and used across globe. It gave them one command centre and power to control the globe from center.

Indians did not use the same to control the globe. They helped the MNCS to control the world and doing so helped them to influence Indians and capture Indian market. This started with advent of new MNC companies like Yahoo, Cisco and Hotmail. The services that was needed by common man in west was needed by rich man in India and people who could afford. Over time poor man was made to use the same too.

On one side, companies provided free access of their services to Indian population. On other side, they setup their own offices and developed web applications at cheaper cost. The burst of internet enabled companies like Cisco, Alcatel and Lucent to sell more hardware to India. They continued to have one challenge, most of Indians did not have access to internet.