Privacy Vs Security In A Big Data World

[Copied article contents Privacy Vs Security In A Big Data World for my reference]

What I do know, however — and I thank him for this — is that Snowden helped bring the discussion of big data privacy and security to the public square — and not just the American public square, but the global one as well. This is a good thing, because in this era of big data, not to mention the Internet of Things, we can no longer relegate this discussion to the privacy freaks and security geeks in the back room. It’s a discussion in which we all should participate.

To understand it better, let’s take a brief look at some of the privacy and security issues in the context of the (big) data lifecycle.

In data security circles, the six stages of the data lifecycle are well known: create, store, use, share, archive, and destroy. While these six stages have a strong foundation in security, an interesting thing to note is the fact that the two privacy-related stages — use and share — are situated squarely in the middle. Is it just a coincidence that privacy is at the heart of the matter?

Create
If data is not collected and/or created, there is no need to secure it. This may seem obvious, but it’s astonishing how many websites and apps forget or disregard this point. They collect it all “just in case” –- with little consideration on how the data may be handled downstream.
Why this matters: Data security begins at the point of creation or collection. Organizations need to be deliberate in the data they request or receive, and individuals should be mindful of the data they’re sharing — whether it’s sensitive data on a financial site or a viral video on YouTube. If this data is not secured, it could end up in the wrong hands.

Store
With the volume of big data being generated these days, it’s not just a question of what data to store, but also how to store it all without blowing the budget. Open-source big data technologies are helping to greatly reduce the cost of data storage, both on-premises and in the cloud.
Why this matters: If an organization creates or collects data, it becomes their responsibility -– not the individuals’ -– to secure and protect it from corruption, destruction, interception, loss, or unauthorized access. Some organizations take this responsibility more seriously than others.

Use
When an individual sets up a new account with an organization through its website/app, the individual is asked to read and agree to the terms of service and/or privacy policy. This legal contract typically defines how the individual’s data will be managed and used inside and outside the organization. Granted, few people read this legalese, but our expectation is that the organization will use our data “responsibly,” and when this usage changes, we expect to be notified.
Why this matters: It’s the usage — not the collection or storage — of data that concerns most people. It’s this stage where individuals want to be in control. For example, they want to set the dial on how public or private their data should be, who can access their data, and whether their data (aggregated or not) can be sold or rented to third parties. In this big data era, when organizations don’t provide this level of privacy control, they risk losing the loyalty and trust of their customers and users.

Share
Organizations continue to share data between internal systems and external partners, but with the advent of social networks and “smart” devices, sharing data has become a public pastime — even to the point of “selfie” narcissism.
Why this matters: On one hand, individuals want control on how their personal data is being used. Yet some of these same individuals show little to no constraint on what personal data they’re sharing. Even though it’s the responsibility of the organization behind the website or app to secure users’ data and respect privacy settings (if they exist), it’s up to the individual to determine what and how much information they’re willing to share. If you put it on the Internet, it’s not a question of if, but when, your information may be used in unintended ways.

Archive
Between big data technologies and the cloud, it’s become relatively cost-effective for organizations to store data for longer periods of time, if not indefinitely. In some cases, regulations stipulate how long certain data will live — like in the US financial and health industries — but, in most cases, the budget and space constraints are being alleviated.
Why this matters: Being able to store more data for longer periods of time at a fraction of the cost is an appealing proposition for organizations. The more exciting proposition, however, is the ability to analyze even more data over greater periods of time to discover new questions, patterns, trends, and anomalies. The gotcha here is: The more data an organization stores and archives, the more data it has to secure.

Destroy
If and when data is tagged for destruction, the question is to what extent. For example, if a website user requests that his account be deleted, what does this mean? Is it just the access to his account/data removed (so that he can request access later if he changes his mind) or does a deletion request trigger the destruction of all his data, including archived data? The answer most likely lies somewhere in between for most organizations.
Why this matters: Regulations and governance policies will dictate the extent to which data may be destroyed for many organizations. The data that does not get destroyed must then be secured. So using the example above, if a website user requests that his account be deleted, and he receives an email notification to that effect, what he doesn’t know is what personal data, if any, still exists in the organization’s systems. He may still be vulnerable to a potential data breach, long after he’s been deleted.
It cuts both ways
While a citizen’s right to privacy and freedom from government surveillance has been top of mind for Edward Snowden, national security has been top of mind for the US government.

And therein lies the rub: security cuts both ways. On one hand, it’s the responsibility of an organization to secure and protect any digital information it collects, stores, and transmits. But on the other hand, our governments are knocking on organizations’ doors demanding access to this protected information — all in the name of preserving national security.

How to Make World Do Work for You

[ Copied from equitymaster email on investment, as it applies to a wider horizon]
Have you ever tried to make a toddler listen when they’re cranky? Trust me, there’s no more difficult job on planet Earth. No matter how hard you try, everything will come to a naught if the kid’s mind is made up.

But what really gets my goat is when the mother comes up, indulges the kind in some sweet talk and voila, the kid is literally eating out of her hands. Instructions, any mother will tell you, never work with a child. You can never expect a child to do what you want him to do. If you really need him to comply, first, be really patient. And second, clearly spell out what’s in it for the kid. Unless the kid sees some benefit in what we are asking him to do, you’ll never get anywhere.
This makes a lot of sense. In fact, it has much wider implications. It will not only help you persuade a child but also achieve success across many walks of life. Joseph Tussman, the prolific American educator, is responsible for one of my all-time favourite quotes:
 
What the pupil must learn, if he learns anything at all, is that the world will do most of the work for you, provided you cooperate with it by identifying how it really works and aligning with those realities. If we do not let the world teach us, it teaches us a lesson.
 
The success mantra
So, what’s the mantra for success? The idea is to find rules that tell us how the world really works and build models around them. You see, there are general principles that have consistently given the desired results going back thousands of years. These are timeless rules that can help us make better decisions. 
If your principles aren’t in sync with reality, you are unlikely to be successful. All you’ll end up with is failure and frustration, like when you give instructions to recalcitrant toddler without explaining what’s in it for him.

Focus on where the real value is

Read this blog contents in article as guidance for existing financial or banking firms interested to  take advantage  of Digital banking. Copied generic part of articles here for my reference( forgot the actual source. Sorry!)

Focus on where the real value is Launching a successful new business requires complete clarity about what its value drivers are. While this seem like an obvious point, we find it is often overlooked. Instead, there is a temptation to copy or replicate existing models.
Constantly test to refine the customer experience Launching a successful new digital-banking business requires a marriage of traditional consumer research and a deep, real-time understanding of the behavior and pain points of individual customers. This means a constant and rapid stream of prototypes starting with the Minimum Viable Product (MVP) and subsequent iterations in order to figure out what will make the customer experience superior across all touch-points. This sort of “real life” testing is critical for identifying what customers actually value as opposed to what they might say they value. It also yields up to 70 percent fewer defects and errors.
Organize for creativity, flexibility, and speed Building a business using a constantly iterative approach requires a way of working that banks typically aren’t used to. There are three areas where a different way of operating needs to be nurtured.

  • Cross-team collaboration. The core group building the digital bank should have a solid understanding of not just the new technology architecture, but also of the bank’s design and brand and the economics of its business model.
  • A ‘garage like’ working environment. While an actual garage isn’t necessary, a physical space that provides a nurturing environment for creative thinking and prototyping is. This means open spaces, plenty of whiteboards and worktables where people can congregate and work together, as well as habits that foster innovation, such as so-called sprints
  • A central ‘control tower’ team. Launching a digital bank is a juggling act, with multiple miniprojects running at the same time. It is the job of the control-tower team to make sure all these projects are coordinated by moving resources to necessary teams quickly or prioritizing initiatives so that timeline targets can be met. The team must work to identify bottlenecks and then either quickly resolve them or refer the problems upward to the CEO or the board.

Create an ecosystem of partnerships Successfully launching a new digital-banking business requires quickly acquiring a critical mass of customers. Two industries with large amounts of digital customers who can help the process are e-commerce marketplaces and telecommunications.
Build a two-speed IT operating model To implement the test-and-learn approach and short release cycles that are so critical for launching and operating a competitive digital bank, two different yet integrated IT systems are needed: the traditional, slower, secure and stable, transaction-focused legacy back end and a rapid, flexible, customer-centric front end.
Get creative with marketing To communicate such distinct selling points cost-effectively, banks must cultivate word-of-mouth recommendations and feedback through social media

Corporate Sustainability

Sustainability begins with a principled approach to doing business.

Born with privilege of father reading  company annual reports where he invests in small amounts and also involved in creating his company annual reports, I started to read Indian company annual reports from age of 12. In 1990s, there was no internet, my father used to have stack of hard bounds (Carries few today also) and started reading. In those days my curiosity made me check for last page and both sides of back cover (in annual report) to see pictures and photos that describe social  and community initiatives of corporate.

On receiving MindTree  annual report last week, I observed there was less or no data on CSR and got surprised.  Today I received sustainability report, that has half number of pages compared to pages in  annual report.  My respect of the company increased in my mind. Learnt about The Ten Principles of the United Nations Global Compact  and you can find more in this article. Copied them for my reference to my blog.

Human Rights

  • Principle 1 Businesses should support and respect the protection of internationally proclaimed human rights;
  • Principle 2 Make sure that they are not complicit in human rights abuses.

Labor

  • Principle 3 Businesses should uphold the freedom of association and the effective recognition of the right to collective bargaining;
  • Principle 4 The elimination of all forms of forced and compulsory labor;
  • Principle 5 The effective abolition of child labor;
  • Principle 6 The elimination of discrimination in respect of employment and occupation.

Environment

  • Principle 7 Businesses should support a precautionary approach to environmental challenges;
  • Principle 8 Undertake initiatives to promote greater environmental responsibility;
  • Principle 9 Encourage the development and diffusion of environmentally friendly technologies.

Anti-corruption

  • Principle 10 Businesses should work against corruption in all its forms, including extortion and bribery. 23

What does a Great Father Do?

Today my daughters wished for Father’s Day. It raised question how should father be to children. Sharing  real life story observed and asking myself “How will I behave with my children in adverse condition where I have no control?” and “Can I be father to my daughters like the father to his son in the story?”

“This is the price you pay for having a great father. You get the wonder, the joy, the tender moments – and you get the tears at the end, too. “

Once upon time, there was father and mother. The retired father had a son and daughter. The son went to engineering college.The son was healthy guy, involved in sports and leading social-curricular activities at college.  The son was college students chairman and coordinated placements also. In his final year, the son got placement in non-IT firm, which was very much aligned to his passion and got placement in IT company.

The son wanted to be in India, with responsibility of taking care his retired parents. When son submitted his medical checkup for non-IT firm, the company send him an email that they are withdrawing the offer and the reason was that he has only one kidney. Yes, one of his kidneys was removed when he was less than 2 years old. The son was broken and could not comprehend the reason, as he never have thought someone would reject him based on physical ailment and have involved in lot of automotive work with passion.  The son went to attend the initial training for IT firm and realized his mind not ready for IT job.

Here comes the real test for father. Your son was confident with his success and for no logical reasons, he has been thrown away from pedestal and how do you  support him. The father supported his son and asked him to do what is best for him. The son joined a job in his area of interest  paying him salary less than Rs 10,000 p.m. This job was a really tight job being on the shop floor and hectic interacting with lot of people.

The father motivated son to appear for GRE/TOEFL. The father enabled son with internet dongle( not common in those days) month, downloaded application, printed them, filled them and got pay order for application fees and send application to colleges, travelling where needed.  The son got admission in to college in USA and had no scholarship.  The father said that he will support with his means. The son quit his job to come and prepare and the father has bought one-way ticket. When you leave job, you lose your medial insurance also.

Here comes the second test for  father. The son plays volleyball every day and one day he slips and his ribs are broken. His visa interview is around. The father and his friends admit him in hospital and they perform minor surgery.  The father acknowledge that falling in sports was natural.  when your children make mistake that impacts their life, as parents we get annoyed and hurt them with our words. The father did not say anything and spend a lot of money, which was limited at his life stage.

Here comes the third test for father: After 2 weeks, they find that surgery has not healed and the surgery needs to be repeated. The father was broken in heart. Visa interview is coming, tickets are booked, son does not have job and son is in hospital, need to spend money, which is  limited and they have to repeat surgery.

The father ran around found an elderly doctor waiting till late night. His only priority was his son life and not him. The elderly doctor said that he would take care  son’s priority and not others. The son may attend visa interview and will not be able to travel on booked date and can travel after week of the booked date.

Before his retirement, the father  bought car and he loved driving  the car. He sold the car and paid for second surgery. He explained others that there is problem with his ears and does not want to drive and hence he is selling his car.  As there was no money to get another fresh ticket, he ran around  and literally pleaded/ requested many people and  with doctor’s  letter was able to get ticket transferred to later date without additional payment.

Readers can see connect between this story and Sujatha‘ story, which I have read also. When I observed this father-son journey, it came across as new age variant and decided to write and wish all fathers Father Day.

Is it fair to ask “Married or Single?”

Read article Married or Single?  posted by Punit Soni. In recruitment process, candidate asked this question”Married or Single?”. I was surprised to see this article. May be I did not expect an article on women rights from Punit Soni (sorry?) and was happy that he was ready to scratch upon the tip of the iceberg. Made me think my IT Career experiences.

Been for more than decade in IT industry, I agree that huge effort is needed from new manager to follow fair approach towards women and differently able people.   Not only large companies can create a fair environment for women, small companies and startups can do also. What is needed is a positive mindset and that comes from positive interactions experienced by male employee early in career in his team or by his supervisor Did they have right experiences that influences them on being manager to  create positive environment for women?

Here my experiences and learning that has helped me to create my perspective towards women employees. sharing the same to say it is possible to create positive mindset.

In my second job, the only Tech Lead I worked for long duration and was friend was women. One day, she had to leave late and she comes by 2-wheeler from south Bangalore. My manager Asif had called for a taxi. On her leaving, he asked her to go home safe in taxi. She went in cab leaving her 2-wheeler. These are times when team worked using desktops and  cab means private card called in advance and we worked in Central Business district. Thanks Asif to help me get right perception to support female employee.

When we went to USA, we had our first daughter born and we were only two of us. Not knowing what to do in new city, I asked my female lead engineer(45 years old) and she was extremely helpful to make me see the responsibility to take care of my wife and my child.   At the same time, She raised question to my  company manger “what will Srini do if baby is born and project is over previous week”. My manger neither gave her proper answer and also came and polished asked “How does she know that your wife is pregnant? I responded that this is our problem and we will handle”.

This lead engineer c and manger shared  how my manger responded and  shared that we take steps to ensure that  your family will not be in zombie state and know what to do. Surprised that how Mid-West Americans supported, contrary to my past wrong belief ” Americans do not care”, both men and women. When baby was born, support received from everyone in Milwaukee, made me humbler and also make me to think of “Pay backwards”what I received. Our manager  attitudes gets developed when we are engineer and the positive it is towards women, it remains positive later. 

In my third job at startup, it was multiple experience. As company, women were treated well. Company bought 4 wheeler and hired driver for company owned 4 wheeler and employed the driver.Officially the driver becomes escort for female employee going late. After Prathibha case, female employees were asked to go home by 8:30 pm and we had a cab at 8:30 pm . Another trip at 7: 30 pm to drop employees to closest bus-stop. My friend Sudhakar showed that we can  balance diversity and being frugal and it is mindset.

On my end, I hired women employee for short term. she was quick learner, well skilled and went for higher studies later.  Thanks to V who  was my first female team member.  She shared how guys behaved, remarks shared without bad intention and why it hurts. My patient listening to her geared myself to hire women employees and understood their needs from manager.

Talent needs to be valued, irrespective of  men or women. My colleagues used to share with women candidates project challenges, late evening call support needed. We conveyed that we will support your well being, we have constraints and still we try our best. Planned their work  to allow them leave home by 8:30 pm or have setup to Work from Home.  We need to answer questions from male employees for this partial support to women.

For my team, there was female candidate for interview for automation engineer and I observed that she was pregnant (already father to 2 children). J performed extremely well in the interview and answered beyond my expectations and her current salary was too low with our offering.  Asking her to wait, I went to my HR(female) & recruitment mgr(child) and said “she is pregnant and she is best fit for role”. They said “Hire for talent”. HR influences in large way in creating culture of organization towards women employees. 

I went back and shared with her our interest and said ” I have seen my wife pregnant.  Are you pregnant?”. She answered yes and asked  how she can travel from Vijaya Nagar in this state. She shared that her company is in ITPL and our office was half the distance compared to her office. I asked next bold question” When is baby due date?”. This date fell 2 weeks before release, there was (>) 5 months from day she can join  I hired her.

This girl learnt learn C# and write code in C# using Visual Studio in first month. She did all this independently with  little supervision with my team lead. Her work impressed me to keep in touch for long time, even after both left company, she went abroad and searched for a job coming back to Bangalore. Today I will hire her again if she looks for a job. Jothi taught me to never look at things like pregnancy in job interview.

In my own startup. S , started as fresher, after an year she got married, she became pregnant too.. We switched off elevator beyond 1 hour in morning and evening. The pregnant employee was permitted to come to the 3rd floor office by elevator. If you are attentive to your actions, you can create exceptions every where. 

When she left of maternity leave, we paid salary across her maternity months.  Thanks to Guru who would keep S’s salary first aside, even in tricky times. When she came back, there was an unfortunate scenario for her to leave. S was given gift and asked to get things right and join back. There was clarity in thought that paying her maternity days was right thing to do as employer. Allowing her to  leave was right thing as that is what is expected from a  parent in her situations.  Being parents, we could relate to S as parents.  Your colleagues drive and influence your approach towards women. 

To end, We see more people beyond women with need on humanity to support them fairly and equally to enable them get a fair life, differently enabled people. As part of my product marketing role in 2nd startup, I proposed an innovative way to make NGOs to use our product. GN, my partner allowed creation of  campaign to provide our product offering free for differently enabled students. While we obtained a lot of learning about our learning software and it opened new opportunities, GN decided to walk the talk by hiring 3 differently enabled people in office. The whole office learnt how to work with them and how we make them feel in our interactions. Thanks GN for humble experience that gave me confidence to work with NGOs and organizations working  with differently able students.

On one side, when companies like SAP offer day care centers and special support for female employees, picture is not rosy across IT employees. These are experiences heard from employees, friends and neighbors.

Female employees, mothers in small companies are scheduled calls at 8:30 PM to 9:30 PM or from 9 PM to 10 PM. What would happen to their children sleeping habit?

Female employees in ITPL areas can be found on BMTC VOLVO buses even after 9 PM when roads get deserted. I see girls who stay in one of our apartments coming late. I am happy with women walking on road in night. If female employee is not ready to care for her safety in Bangalore famous for its Pratibha case, how will it strike to manager (male) to think of female employee needs? 

Both male and female employees in India attend calls between 6:30 PM and 8:30 PM. Assume that these Indians can attend from home, when do they spend time with their children? Mostly, you would see their American colleagues would do best to have all things to  protects his personal time. Do Indians not need personal time?.

I know houses where fathers lock their rooms because child would not allow them to take calls. when my girls were young, they come and sit on my lap across client call. I used to inform that she is listening and my daughters rarely shouted/screamed. May be a rare case. Today in home calls, when people ask to ON webcams, I do not or at times I hear that I look dull, I ignore and focus on agenda.  I and my daughter share study room. .

To note, at closing time of call at 9:30 PM, lead/manager asks team member to send minutes of meeting  or send email with some details or perform a small change and wants the same immediate, sharing that would help them be more productive. My question is ” why they fail to understand that employee took call in personal time? How fair it is of them to assign work and ask for  completion in personal time?

How to test application implementing ML algorithm?

To perform testing of software programs, one arrives at a set of tests steps to test programs and test data to be provided at each of testable step and the expected output from the program based on the test data and test step. If the actual output from the program is same as expected output, we declare that program is functioning fine.  The working of the program gets tested for correctness for boundary and exception scenarios of both program and data input.

Having spec algorithm’s, coded algorithm’s, unit tested algorithm and tested them as part of application in my earlier days, I want to understand how people test mobile learning programs. This is my current understanding which I want to improve.

Coming to software testing of machine learning program, directly applying conventional software engineering process may not work. It is challenge to detect errors, faults and defects in machine learning program that takes arbitrary input to generate program’s output and to determine whether the program’s output is correct or reliable for the data inputs. Are ML programs non-testable?

Should testing of machine learning program focus less on whether ML algorithm learns well and focus more on whether application using the algorithm implements the specification and fulfills the user’s expectations?

First, start to understand the problem domain and suitability of algorithm in the problem context based on potential range of data inputs arriving in real time, in terms of real world data sets. Thinking of data sets can start with following data-set characteristics. Small vs large, repeating vs non-repeating values, missing vs non-missing attribute values, repeating vs non-repeating attribute labels,  predictable vs non-predictable attribute values, attributes that take non-negative values only , attributes that can also take negative value and the precision required for floating point numbers.

Second, test working of algorithm and third is to test algorithm providing data inputs.

  • Are you implementing algorithm? Design a series of primitive tests for various sub-parts of the algorithms, and an end-to-end test testing the final output or algorithm behavior.
  • Are you making use of some algorithm? Understand the algorithm and required validation for user inputs to ensure getting best possible results and how to arrive in the problem context, whether the algorithm results are sensible or not.
  • Check  upper bound reports on time and space used by the algorithm and get a measure of efficiency in terms of size or complexity of its input (Big O notation).

Think in terms of unit tests and regression tests for machine learning programs.

  • Add unit tests to your code and have approximate testing of your expected results
  • Create multiple data-sets with different difficulty levels like easy, difficult and adversarial. Whenever code changes to add a feature or fix a bug, run code against all of these data-sets to ensure that expected outputs lie in a reasonable error range and do not break existing functionality.

Arrive at criteria to determine meaning of correctness, working with domain specialists.

Discuss, Decide and determine margin of errors or correctness percentage beforehand to testing machine learning program. For example, if program interprets 75% of test data correctly, the programs is considered to be good enough. Remember that it might not be possible to demand test validation of 100% correctness as the intent of machine learning is to tolerate ambiguity.

Testing would benefit with software engineers ability to provide a data set generator, tools that would help to compare the output results and their correctness based on the data inputs.  You need to have methods to capture and view trace options that are inserted in to the ML program and tools to analyse traces to debug, test and validate intermediate results in specific steps of the algorithm.