Join IoT Central | Join our LinkedIn Group | Post on IoT Central


analytics (21)

Looking Back at 2020: A Year of Digital Resiliency

Posted by Rob Tiffany on January 4, 2021 at 16:27 in Software, OSes, and Programming

When I think about the things that held the planet together in 2020, it was digital experiences delivered over wireless connectivity that made remote things local.

While heroes like doctors, nurses, first responders, teachers, and other essential personnel bore the brunt of the COVID-19 response, billions of people around the world found themselves cut off from society. In order to keep people safe, we were physically isolated from each other. Far beyond the six feet of social distancing, most of humanity weathered the storm from their homes.

And then little by little, old things we took for granted, combined with new things many had never heard of, pulled the world together. Let’s take a look at the technologies and trends that made the biggest impact in 2020 and where they’re headed in 2021:

The Internet

The global Internet infrastructure from which everything else is built is an undeniable hero of the pandemic. This highly-distributed network designed to withstand a nuclear attack performed admirably as usage by people, machines, critical infrastructure, hospitals, and businesses skyrocketed. Like the air we breathe, this primary facilitator of connected, digital experiences is indispensable to our modern society. Unfortunately, the Internet is also home to a growing cyberwar and security will be the biggest concern as we move into 2021 and beyond. It goes without saying that the Internet is one of the world’s most critical utilities along with water, electricity, and the farm-to-table supply chain of food.

Wireless Connectivity

People are mobile and they stay connected through their smartphones, tablets, in cars and airplanes, on laptops, and other devices. Just like the Internet, the cellular infrastructure has remained exceptionally resilient to enable communications and digital experiences delivered via native apps and the web. Indoor wireless connectivity continues to be dominated by WiFi at home and all those empty offices. Moving into 2021, the continued rollout of 5G around the world will give cellular endpoints dramatic increases in data capacity and WiFi-like speeds. Additionally, private 5G networks will challenge WiFi as a formidable indoor option, but WiFi 6E with increased capacity and speed won’t give up without a fight. All of these developments are good for consumers who need to stay connected from anywhere like never before.

Web Conferencing

With many people stuck at home in 2020, web conferencing technology took the place of traveling to other locations to meet people or receive education. This technology isn’t new and includes familiar players like GoToMeeting, Skype, WebEx, Google Hangouts/Meet, BlueJeans, FaceTime, and others. Before COVID, these platforms enjoyed success, but most people preferred to fly on airplanes to meet customers and attend conferences while students hopped on the bus to go to school. In 2020, “necessity is the mother of invention” took hold and the use of Zoom and Teams skyrocketed as airplanes sat on the ground while business offices and schools remained empty. These two platforms further increased their stickiness by increasing the number of visible people and adding features like breakout rooms to meet the demands of businesses, virtual conference organizers, and school teachers. Despite the rollout of the vaccine, COVID won’t be extinguished overnight and these platforms will remain strong through the first half of 2021 as organizations rethink where and when people work and learn. There’s way too many players in this space so look for some consolidation.

E-Commerce

“Stay at home” orders and closed businesses gave e-commerce platforms a dramatic boost in 2020 as they took the place of shopping at stores or going to malls. Amazon soared to even higher heights, Walmart upped their game, Etsy brought the artsy, and thousands of Shopify sites delivered the goods. Speaking of delivery, the empty city streets became home to fleets FedEx, Amazon, UPS, and DHL trucks bringing packages to your front doorstep. Many retail employees traded-in working at customer-facing stores for working in a distribution centers as long as they could outperform robots. Even though people are looking forward to hanging out at malls in 2021, the e-commerce, distribution center, delivery truck trinity is here to stay. This ball was already in motion and got a rocket boost from COVID. This market will stay hot in the first half of 2021 and then cool a bit in the second half.

Ghost Kitchens

The COVID pandemic really took a toll on restaurants in the 2020, with many of them going out of business permanently. Those that survived had to pivot to digital and other ways of doing business. High-end steakhouses started making burgers on grills in the parking lot, while takeout pizzerias discovered they finally had the best business model. Having a drive-thru lane was definitely one of the keys to success in a world without waiters, busboys, and hosts. “Front of house” was shut down, but the “back of house” still had a pulse. Adding mobile web and native apps that allowed customers to easily order from operating “ghost kitchens” and pay with credit cards or Apple/Google/Samsung Pay enabled many restaurants to survive. A combination of curbside pickup and delivery from the likes of DoorDash, Uber Eats, Postmates, Instacart and Grubhub made this business model work. A surge in digital marketing also took place where many restaurants learned the importance of maintaining a relationship with their loyal customers via connected mobile devices. For the most part, 2021 has restauranteurs hoping for 100% in-person dining, but a new business model that looks a lot like catering + digital + physical delivery is something that has legs.

The Internet of Things

At its very essence, IoT is all about remotely knowing the state of a device or environmental system along with being able to remotely control some of those machines. COVID forced people to work, learn, and meet remotely and this same trend applied to the industrial world. The need to remotely operate industrial equipment or an entire “lights out” factory became an urgent imperative in order to keep workers safe. This is yet another case where the pandemic dramatically accelerated digital transformation. Connecting everything via APIs, modeling entities as digital twins, and having software bots bring everything to life with analytics has become an ROI game-changer for companies trying to survive in a free-falling economy. Despite massive employee layoffs and furloughs, jobs and tasks still have to be accomplished, and business leaders will look to IoT-fueled automation to keep their companies running and drive economic gains in 2021.

Streaming Entertainment

Closed movie theaters, football stadiums, bowling alleys, and other sources of entertainment left most people sitting at home watching TV in 2020. This turned into a dream come true for streaming entertainment companies like Netflix, Apple TV+, Disney+, HBO Max, Hulu, Amazon Prime Video, Youtube TV, and others. That said, Quibi and Facebook Watch didn’t make it. The idea of binge-watching shows during the weekend turned into binge-watching every season of every show almost every day. Delivering all these streams over the Internet via apps has made it easy to get hooked. Multiplayer video games fall in this category as well and represent an even larger market than the film industry. Gamers socially distanced as they played each other from their locked-down homes. The rise of cloud gaming combined with the rollout of low-latency 5G and Edge computing will give gamers true mobility in 2021. On the other hand, the video streaming market has too many players and looks ripe for consolidation in 2021 as people escape the living room once the vaccine is broadly deployed.

Healthcare

With doctors and nurses working around the clock as hospitals and clinics were stretched to the limit, it became increasingly difficult for non-COVID patients to receive the healthcare they needed. This unfortunate situation gave tele-medicine the shot in the arm (no pun intended) it needed. The combination of healthcare professionals delivering healthcare digitally over widespread connectivity helped those in need. This was especially important in rural areas that lacked the healthcare capacity of cities. Concurrently, the Internet of Things is making deeper inroads into delivering the health of a person to healthcare professionals via wearable technology. Connected healthcare has a bright future that will accelerate in 2021 as high-bandwidth 5G provides coverage to more of the population to facilitate virtual visits to the doctor from anywhere.

Working and Living

As companies and governments told their employees to work from home, it gave people time to rethink their living and working situation. Lots of people living in previously hip, urban, high-rise buildings found themselves residing in not-so-cool, hollowed-out ghost towns comprised of boarded-up windows and closed bars and cafés. Others began to question why they were living in areas with expensive real estate and high taxes when they not longer had to be close to the office. This led to a 2020 COVID exodus out of pricey apartments/condos downtown to cheaper homes in distant suburbs as well as the move from pricey areas like Silicon Valley to cheaper destinations like Texas. Since you were stuck in your home, having a larger house with a home office, fast broadband, and a back yard became the most important thing. Looking ahead to 2021, a hybrid model of work-from-home plus occasionally going into the office is here to stay as employees will no longer tolerate sitting in traffic two hours a day just to sit in a cubicle in a skyscraper. The digital transformation of how and where we work has truly accelerated.

Data and Advanced Analytics

Data has shown itself to be one of the world’s most important assets during the time of COVID. Petabytes of data has continuously streamed-in from all over the world letting us know the number of cases, the growth or decline of infections, hospitalizations, contact-tracing, free ICU beds, temperature checks, deaths, and hotspots of infection. Some of this data has been reported manually while lots of other sources are fully automated from machines. Capturing, storing, organizing, modeling and analyzing this big data has elevated the importance of cloud and edge computing, global-scale databases, advanced analytics software, and the growing importance of machine learning. This is a trend that was already taking place in business and now has a giant spotlight on it due to its global importance. There’s no stopping the data + advanced analytics juggernaut in 2021 and beyond.

Conclusion

2020 was one of the worst years in human history and the loss of life was just heartbreaking. People, businesses, and our education system had to become resourceful to survive. This resourcefulness amplified the importance of delivering connected, digital experiences to make previously remote things into local ones. Cheers to 2021 and the hope for a brighter day for all of humanity.

Read more…

IoT ,Smart Cities and Digital Transformation are the Keys Ahead

Posted by Raj Kosaraju on March 7, 2018 at 8:06 in Platforms

Technologists and analysts are on a path to discovery, obtaining answers on how technology and the data collected can make our cities more efficient and cost effective. 

While IoT may be seen as another buzzword at the moment, companies like SAP, Cloud Sigma, Net Atlantic and Amazon Web Services are working to make sure that for businesses, IoT is a reality. It’s companies with this willingness to change, adopt and invent that will win the new economy. Mobile phones, online shopping, social networks, electronic communication, GPS and instrumented machinery all produce torrents of data as a by-product of their ordinary operations. Most companies want their platform to be the foundation of everything it does, whether it is with big data, data analytics, IoT or app development. The same  rub off phenomenon was emulated in Latin American countries  like Brazil, Argentina, Mexico and European countries like Brussels, Italy,  Germany, Denmark , Poland and Prague in recent times.

It is important to realize that technology is exploding before our very eyes, generating unprecedented opportunities. With easy access to cheap cloud services, smarter people came up with these platforms, and it has fundamentally changed businesses and created new ways of working. Mobile cannot be an afterthought. It needs to be integrated in everything you do and positioned at the forefront of your strategy. You have no valid reason to avoid migrating to the cloud. Cloud provides a ubiquitous, on-demand, broad network with elastic resource pooling. It’s a self-configurable, cost-effective computing and measured service. On the application side, cloud computing helps in adopting new capabilities, meeting the costs to deploy, employing viable software, and maintaining and training people on enterprise software. If enterprises want to keep pace, they need to emulate the architectures, processes and practices of these exemplary cloud providers.

One of the main factors of contributing value additions is the concept of a Smart City which is described as one that uses digital technologies or information and communication technologies to enhance the quality and performance of urban services, to reduce costs and resource consumption and to engage more effectively and actively with its citizens. We will interact and get information from these smart systems using our smartphones, watches and other wearables, and crucially, the machines will also speak to each other.The idea is to embed the advances in technology and data collection which are making the Internet of Things (IoT) a reality into the infrastructures of the environments where we live. We will interact and get information from these smart systems using our smartphones, watches and other wearables, and crucially, the machines will also speak to each other. Technologists and analysts are on a path to discovery, obtaining answers on how technology and the data collected can make our cities more efficient and cost effective. The current model adopted for IoT is to attract businesses to develop software and hardware applications in this domain. The model also encourages businesses to put their creativity to use for the greater good, making cities safer, smarter and more sustainable.

A few years ago like many others  I predicted  that Business models will be shaped by analytics, data and the cloud. Moreover, the IoT is deeply tied in with data, analytics and cloud to enable them and to improve solutions. The key goal is to ensure there is value to both customers and businesses. You can effectively put this strategy into action and build a modern data ecosystem that will transform your data into actionable insights.  

Till we meet next time...

Best,

Raj Kosaraju

CIO 

 

Read more…

Do you want to hire a Data Scientist?

Posted by Sandeep raut on August 6, 2017 at 14:06
As mentioned by Tom Davenport few years back, Data Scientist is still a hottest job of century.
Data scientists are those elite people who solve business problems by analyzing tons of data and communicate the results in a very compelling way to senior leadership and persuade them to take action.
They have the critical responsibility to understand the data and help business get more knowledgeable about their customers.
The importance of Data Scientists has rose to top due to two key issues:
·     Increased need & desire among businesses to gain greater value from their data to be competitive
·     Over 80% of data/information that businesses generate and collect is unstructured or semi-structured data that need special treatment
So it is extremely important to hire a right person for the job.Requirements for being a data scientist are pretty rigorous, and truly qualified candidates are few and far between.
Data Scientists are very high in demand, hard to attract, come at a very high cost so if there is a wrong hire then it’s really more frustrating. 
Here are some guidelines for checking them:
·     Check the logical reasoning ability
·     Problem solving skills
·     Ability to collaborate & communicate with business folks
·     Practical experience on collaborating  Big Data tools
·     Statistical and  machine learning experience
·     Should be able to describe their projects very clearly where they have solved business problems
·     Should be able to tell story from the data
·     Should know the latest of  cognitive computing,  deep learning
I have seen smartest data scientists in my career who do the best job best but cannot communicate the results to senior leaders effectively. Ideally they should know the data in depth and can explain its significance properly. Data visualizations comes very handy at this stage.
Today with  digital disrupting every field it has an impact on data science also.
Gartner has called this new breed as citizen data scientists. Their primary job function is outside  analytics, they don’t know much about statistics but can work on ready to use algorithms available in APIs like Watson, Tensor flow, Azure and other well-known tools.
The good data scientist can make use of them to spread the awareness and expand their influence.
It has become more important to hire a right data scientist as they will show you the results which may make or break the company.
Read more…

How to measure Digital Transformation maturity?

Posted by Sandeep raut on July 15, 2017 at 12:25 in OSes
The digital revolution has created significant opportunities and threats for every industry. Companies that cannot or do not make significant changes faster to their business model in response to a disruption are unlikely to  survive
It is extremely important to do digital maturity assessment before embarking on  digital transformation.
Digital leaders must respond to the clear and present threat of digital disruption by transforming their businesses. They must embed digital capabilities into the very heart of their business, making digital a core competency, not a bolt-on. Creating lasting transformative digital capabilities requires you to build a  customer-centric culture within your organization.
This requires new capabilities that organizations need to acquire and develop which include disruptive technologies like  Big Data, Analytics,  Internet of Things, newer business models.
Digital maturity model measures readiness of the organization to attain higher value in digital  customer engagement, digital operations or digital services. It helps in incremental adoption of digital technologies and processes to drive competitive strategies, greater operationally agility and respond to rapidly changing market conditions.
Business can use the maturity model to define the roadmap, measuring progress on the milestones.
The levels of maturity can be defined as per multiple reports available and

adopt the ones which makes more sense to your business.

·     Level 1 : Project based solutions are developed for a particular problem, no integration to home grown systems, unaware of risks and opportunities
·     Level 2 : Departmentalized projects but still not known to organization, little integration
·     Level 3 : Solutions are shared between the departments for a common business problem, better integration
·     Level 4 : Organization wide efforts of digital, highly integrated, adaptive culture for  fail fast  and improve
·     Level 5 : Driven by CXOs, customer centric and complete transformation changes happen to organization
Here are the 7 categories on which business should ask questions to all the stakeholders to gauge the maturity of Digital Transformation and identify the improvement and priorities.
1.   Strategy & Roadmap - how the business operates or transforms to increase its competitive advantage through digital initiatives which are embedded within the overall business strategy
2.   Customer – Are you providing experience to customers on their preferred channels, online, offline, anytime on any device
3.   Technology – Relevant tools and technologies to make data available across all the systems
4.   Culture – Do you have the organization structure and culture to drive the digital top down
5.   Operations – Digitizing & automating the processes to enhance business efficiency and effectiveness.
6.   Partners – Are you utilizing right partners to augment your expertise
7.   Innovation – How employees are encouraged to bring the continuous innovation to how they serve the customers
Finally you know when you are digital transformed?
·             When there is nobody having “Digital” in their title
·             There is no marketing focused on digital within the organization
·             There is no separate digital strategy than company’s business strategy
Read more…

5 Ways to Improve the Model Accuracy of Machine Learning

Posted by Sandeep raut on April 30, 2017 at 4:12 in OSes
Today we are into  digital age, every business is using  big data and machine learning to effectively target users with messaging in a language they really understand and push offers, deals and ads that appeal to them across a range of channels.
With exponential growth in data from people and &  internet of things, a key to survival is to use machine learning & make that data more meaningful, more relevant to enrich customer experience.
Machine Learning can also wreak havoc on a business if improperly implemented. Before embracing this technology, enterprises should be aware of the ways machine learning can fall flat. Data scientists have to take extreme care while developing these machine learning models so that it generate right insights to be consumed by business.
Here are 5 ways to improve the accuracy & predictive ability of machine learning model and ensure it produces better results.
·       Ensure that you have variety of data that covers almost all the scenarios and not biased to any situation. There was a news in early pokemon go days that it was showing only white neighborhoods. It’s because the creators of the algorithms failed to provide a diverse training set, and didn't spend time in these neighborhoods. Instead of working on a limited data, ask for more data. That will improve the accuracy of the model.
·       Several times the data received has missing values. Data scientists have to treat outliers and missing values properly to increase the accuracy. There are multiple methods to do that – impute mean, median or mode values in case of continuous variables and for categorical variables use a class. For outliers either delete them or perform some transformations.
·       Finding the right variables or features which will have maximum impact on the outcome is one of the key aspect. This will come from better domain knowledge, visualizations. It’s imperative to consider as many relevant variables and potential outcomes as possible prior to deploying a machine learning algorithm.
·        Ensemble models is combining multiple models to improve the accuracy using bagging, boosting. This ensembling can improve the predictive performance more than any single model. Random forests are used many times for ensembling.
·       Re-validate the model at proper time frequency. It is necessary to score the model with new data every day, every week or month based on changes in the data. If required rebuild the models periodically with different techniques to challenge the model present in the production.
There are some more ways but the ones mentioned above are foundational steps to ensure model accuracy.
Machine learning gives the super power in the hands of organization but as mentioned in the Spider Man movie – “With great power comes the great responsibility” so use it properly.
Read more…

A to Z of Analytics

Posted by Sandeep raut on April 16, 2017 at 4:02 in OSes

Analytics has taken world by storm & It it the powerhouse for all the digital transformation happening in every industry.

Today everybody is generating tons of data – we as consumers leaving digital footprints on social media, IoT generating millions of records from sensors,  Mobile phones are used from morning till we sleep. All these variety of data formats are stored in  Big Data platform. But only storing this data is not going to take us anywhere unless analytics is applied on it. Hence it is extremely important to close the loop with Analytics insights.
Here is my version of A to Z for Analytics:
Artificial Intelligence: AI is the capability of a machine to imitate intelligent human behavior. BMW, Tesla, Google are using AI for self-driving cars. AI should be used to solve real world tough problems like climate modeling to disease analysis and betterment of humanity.
Boosting and Bagging: it is the technique used to generate more accurate models by ensembling multiple models together
Crisp-DM: is the cross industry standard process for data mining.  It was developed by a consortium of companies like SPSS, Teradata, Daimler and NCR Corporation in 1997 to bring the order in developing analytics models. Major 6 steps involved are business understanding, data understanding, data preparation, modeling, evaluation and deployment.
Data preparation: In analytics deployments more than 60% time is spent on data preparation. As a normal rule is garbage in garbage out. Hence it is important to cleanse and normalize the data and make it available for consumption by model.
Ensembling: is the technique of combining two or more algorithms to get more robust predictions. It is like combining all the marks we obtain in exams to arrive at final overall score. Random Forest is one such example combining multiple decision trees.
Feature selection: Simply put this means selecting only those feature or variables from the data which really makes sense and remove non relevant variables. This uplifts the model accuracy.
Gini Coefficient: it is used to measure the predictive power of the model typically used in credit scoring tools to find out who will repay and who will default on a loan.
Histogram: This is a graphical representation of the distribution of a set of numeric data, usually a vertical bar graph used for exploratory analytics and data preparation step.
Independent Variable: is the variable that is changed or controlled in a scientific experiment to test the effects on the dependent variable like effect of increasing the price on Sales.
Jubatus: This is online Machine Learning Library covering Classification, Regression, Recommendation (Nearest Neighbor Search), Graph Mining, Anomaly Detection, Clustering
KNN: K nearest neighbor algorithm in  Machine Learning used for classification problems based on distance or similarity between data points.
Lift Chart: These are widely used in campaign targeting problems, to determine which decile can we target customers for a specific campaign. Also, it tells you how much response you can expect from the new target base.
Model: There are more than 50+ modeling techniques like regressions, decision trees, SVM, GLM, Neural networks etc present in any technology platform like SAS Enterprise miner, IBM SPSS or R. They are broadly categorized under supervised and unsupervised methods into classification, clustering, association rules.
Neural Networks: These are typically organized in layers made up by nodes and mimic the learning like brain does. Today  Deep Learning is emerging field based on deep neural networks.
 
Optimization: It the Use of simulations techniques to identify scenarios which will produce best results within available constraints e.g. Sale price optimization, identifying optimal Inventory for maximum fulfillment & avoid stock outs
PMML: this is xml base file format developed by data mining group to transfer models between various technology platforms and it stands for predictive model markup language.
Quartile: It is dividing the sorted output of model into 4 groups for further action.
R: Today every university and even corporates are using R for statistical model building. It is freely available and there are licensed versions like Microsoft R. more than 7000 packages are now available at disposal to data scientists.
Sentiment Analytics: Is the process of determining whether an information or service provided by business leads to positive, negative or neutral human feelings or opinions. All the consumer product companies are measuring the sentiments 24/7 and adjusting there marketing strategies.
Text Analytics: It is used to discover & extract meaningful patterns and relationships from the text collection from social media site such as Facebook, Twitter, Linked-in, Blogs, Call center scripts.
Unsupervised Learning: These are algorithms where there is only input data and expected to find some patterns. Clustering & Association algorithms like k-menas & apriori are best examples.
Visualization: It is the method of enhanced exploratory data analysis & showing output of modeling results with highly interactive statistical graphics. Any model output has to be presented to senior management in most compelling way. Tableau, Qlikview, Spotfire are leading visualization tools.
What-If analysis: It is the method to simulate various business scenarios questions like what if we increased our marketing budget by 20%, what will be impact on sales? Monte Carlo simulation is very popular.
What do think should come for X, Y, Z?
Read more…

What is the difference between Consumer IoT and Industrial IoT (IIoT)?

Posted by Sandeep raut on February 19, 2017 at 5:27 in OSes
Internet of Things (IoT) began as an emerging trend and has now become one of the key element of Digital Transformationthat is driving the world in many respects.
If your thermostat or refrigerator is connected to the Internet, then it is part of the consumer IoT.  If your factory equipment have sensors connected to internet, then it is part of  Industrial  IoT(IIoT).
IoT has an impact on end consumers, while IIoT has an impact on industries like Manufacturing, Aviation, Utility, Agriculture, Oil & Gas, Transportation, Energy and Healthcare.
IoT refers to the use of "smart" objects, which are everyday things from cars and home appliances to athletic shoes and light switches that can connect to the Internet, transmitting and receiving data and connecting the physical world to the digital world.
IoT is mostly about human interaction with objects. Devices can alert users when certain events or situations occur or monitor activities:
·       Google Nest sends an alert when temperature in the house dropped below 68 degrees
·       Garage door sensors alert when open
·       Turn up the heat and turn on the driveway lights a half hour before you arrive at your home
·       Meeting room that turns off lights when no one is using it
·       A/C switch off when windows are open
IIoT on the other hand, focus more workers safety, productivity & monitors activities and conditions with remote control functions ability:
·        Drones to monitor oil pipelines
·       Sensors to monitor Chemical factories, drilling equipment, excavators, earth movers
·       Tractors and sprayers in agriculture
·        Smart cities might be a mix of commercial and IIoT.
IoT is important but not critical while IIoT failure often results in life-threatening or other emergency situations.
IIoT provides an unprecedented level of visibility throughout the supply chain. Individual items, cases, pallets, containers and vehicles can be equipped with auto identification tags and tied to GPS-enabled connections to continuously update location and movement.
IoT generates medium or high volume of data while IIoT generates very huge amounts of data (A single turbine compressor blade can generate more than 500GB of data per day) so includes  Big Data, Cloud computing,  machine learning as necessary computing requirements.
In future, IoT will continue to enhance our lives as consumers while IIoT will enable efficient management of entire supply chain.
Read more…

Digital Transformation and high-tech Robo-Advisor - do you need one?

Posted by Sandeep raut on February 10, 2017 at 3:36 in OSes
How many times you have listened to the advice of your friend/colleague or someone you know, to invest in stock market? Many people have gained and lost their fortune with this guess work and now younger generation is more scared to hand over their hard earned money to someone for investing.
Until recently, you had 2 options for investments - either hire a human financial advisor or do it yourself. Human advisors charge substantial fees starting minimum 1% of value of assets to manage your portfolios. Do it yourself option requires lot of time and energy and you may lose your money due to result of overtrading, panic-selling during downturns, and trying to time the market as the issue for many individuals is they aren’t cut out to go it alone
This is where robo-advisors have scored more over humans.
A robo-advisor is an online, automated wealth management service based on  data science algorithms with no or minimal human interventions that allocate, deploy and rebalance(spreading your money in stocks, mutual funds, bonds to balance risks) your investments.
The robo-advisor industry is in its infancy. Online life is migrating from persona desktop computing to laptops to tablets and finally to  mobile.
Here are some of the advantages of using a robo-advisor:
·       Cheaper fees or free compared to traditional financial advisors
·       Automatic diversification into various options
·       Easy online access as we all are accustomed to shiny apps on mobile
·       Safer than picking your own stocks
·       You don’t need a degree in finance to understand the recommendations.
Big data and advanced  analytics can help broaden the scope of robo-advice dramatically, incorporating financial planning into broader retirement planning, tax planning, vacation savings, higher education planning.
Robo-Advisors have typically targeted millennials segment because these young investors want to save & multiple money faster and often don't have enough patience & wealth to warrant the attention and interest of a human advisor.
High Net worth Individuals also think, online and automated investment tools can positively affect their wealth manager's advice and decision-making.
Overall, robo-advisors provide a good  user experience with latest  digital technologies such as slick apps and fancy interfaces. These platforms make sure that they fit right in with your daily online browsing,  and are great options for novice investors who are just starting out and want to dip their toes in the world of investments, or for people with a simple financial plan who just need an affordable, straightforward place to start their retirement plans
Wealthfront & Betterment are two popular commercial fee based robo-advisors available today. In the Free category WiseBanyan & CharlesSchwab are making the ground.
But it won’t be long before Amazon, Google, Facebook and Apple get in on the robo-advisor industry.
Robo advice is certainly here to stay, and it has its place in the wealth management landscape of tomorrow. But what's missing most, with robo-advisers is the personal touch.  In this age of  hyper-personalization, the lack of a human element is one area where robo-advisors may fall short.
The robo-advisor can't replace a trusted age old adviser, your elders have worked with, who lives nearby and can rush right over in case of need, who knows you and your family.

With the pace of improvement that  Artificial Intelligence and  machine learning bringing up, robo-advice has the potential to become highly personalized and specific over time.
Read more…

See this simple introduction to Natural Language Processing (NLP)

Posted by Sandeep raut on January 21, 2017 at 19:23 in OSes
Today, with  Digitization of everything, 80 percent the data being created is unstructured. 
Audio, Video, our social footprints, the data generated from conversations between customer service reps, tons of legal document’s texts processed in financial sectors are examples of unstructured data stored in  Big Data.
Organizations are turning to natural language processing (NLP) technology to derive understanding from the myriad of these unstructured data available online and in call-logs.
Natural language processing (NLP) is the ability of computers to understand human speech as it is spoken. NLP is a branch of  artificial intelligence that has many important implications on the ways that computers and humans interact.  Machine Learning has helped computers parse the ambiguity of human language.
Apache OpenNLP, Natural Language Toolkit(NLTK), Stanford NLP are various open source NLP libraries used in real world application below.
Here are multiple ways NLP is used today:
The most basic and well known application of NLP is Microsoft Word spell checking.
Text analysis, also known as  sentiment analytics is a key use of NLP. Businesses are most concerned with comprehending how their customers feel emotionally adn use that data for betterment of their service.
Email filters are another important application of NLP. By analyzing the emails that flow through the servers, email providers can calculate the likelihood that an email is spam based its content by using Bayesian or Naive based spam filtering.
Call centers representatives engage with customers to hear list of specific complaints and problems. Mining this data for sentiment can lead to incredibly actionable intelligence that can be applied to product placement, messaging, design, or a range of other use cases.
Google and Bing and other search systems use NLP to extract terms from text to populate their indexes and to parse search queries.
Google Translate applies machine translation technologies in not only translating words, but in understanding the meaning of sentences to provide a true translation.
Many important decisions in financial markets use NLP by taking plain text announcements, and extracting the relevant info in a format that can be factored into algorithmic trading decisions. E.g. news of a merger between companies can have a big impact on trading decisions, and the speed at which the particulars of the merger, players, prices, who acquires who, can be incorporated into a trading algorithm can have profit implications in the millions of dollars.
Since the invention of the typewriter, the keyboard has been the king of human-computer interface. But today with voice recognition via  virtual assistants, like Amazon’s Alexa, Google’s Now, Apple’s Siri and Microsoft’s Cortana respond to vocal prompts and do everything from finding a coffee shop to getting directions to our office and also tasks like turning on the lights in home, switching the heat on etc. depending on how digitized and wired-up our life is.
Question Answering - IBM Watson is the most prominent example of question answering via information retrieval that helps guide in various areas like healthcare, weather, insurance etc.
Therefore it is clear that Natural Language Processing takes a very important role in new machine human interfaces. It’s an essential tool for leading-edge analytics & is the near future.
Read more…

This is how Analytics is changing the game of Sports!!

Posted by Sandeep raut on January 14, 2017 at 20:27 in OSes
Analytics and  Big Data have disrupted many industries, and now they are on the edge of scoring major points in sports. Over the past few years, the world of sports has experienced an explosion in the use of analytics
Till few years back experience, gut feelings, and superstition have traditionally shaped the decision making process in sports.
It is first started with Oakland Athletics' General Manager, Billy Beane, who applied analytics for selecting right players. This was the first known use of  statistics and data to make decisions in professional sports.
Today, every major professional sports team either has an analytics department or an analytics expert on staff.  From coaches and players to front offices and businesses, analytics can make a difference in scoring touchdowns, signing contracts or preventing injuries.
Big name organizations such as the Chicago Cubs, and Golden State Warriors are realizing that this is the future of sports and it is in their best interest to ride the wave while everyone else is trying to learn how to surf.
Golden State Warriors, have similarly used big data sets to help owners and coaches recruit players and execute game plans.
SportVu has six cameras installed in the NBA arenas to track the movements of every player on the court and the basketball 25 times per second. The data collected provides a plethora of innovative statistics based on speed, distance, player separation and ball possession to improve next games.
Adidas miCoach app works by having players attach a  wearable device to their jerseys. Data from the device shows the coach who the top performers are and who needs rest. It also provides real-time stats on each player, such as speed, heart rate and acceleration.
Patriots developed a  mobile app called Patriots Game Day Live, available to anyone attending a game at Gillette Stadium. With this app, they are trying to predict the wants and needs of fans, special content to be delivered, in-seat concession ordering and bathroom wait times.
FiveThirtyEight.com, provides details into more than just baseball coverage. It has over 20 journalists crunching numbers for fans to gain a better understanding of an upcoming game, series or season.
Motus’ new sleeves for tracking a pitcher’s throwing motion, measuring arm stress, speed and shoulder rotation. The advanced data generated from this increases a player’s health, performance and career. Experts can now predict with greater confidence if and when a pitcher with a certain throwing style will get injured.

In the recent Cricket world cup, every team had its own team of Data Analysts. They used various technologies like  Cloud Platform and visualizations to predict scores, player performance, player profiles and more. Around 40 years’ worth of Cricket World Cup data is being mined to produce insights that enhances the viewer's experience. 
Analytics can advance the sports fans' experience as teams and ticket vendors compete with the at-home experience -- the better they know their fans, the better they can cater to them.
This collection of data is also used for internet ads, which can help with the expansion and growth of your organization through social media platforms or websites. 
  • What would be the most profitable food served at the concession stand?
  • What would be the best prices to sell game day tickets?
  • Determine which player on the team is the most productive?
  • Which players in the draft will become all-stars, and which ones will be considered role players?
  • Understand the fans behavior at the stadium via their app and push relevant information accordingly.
In this  Digital age, Analytics are the present and future of professional sports. Any team that does not apply them to the fullest is at a competitive disadvantage.
Read more…

Do you know what is powerful real-time analytics?

Posted by Sandeep raut on December 30, 2016 at 18:59 in OSes
In the  Digital age today, world has become smaller and faster. 
Global audio & video calls which were available only in corporate offices, are now available to common man on the smartphone.
Consumers have more information of the products and comparison than the manufactures at any time, any place, and any device.
Gone are the days, when organizations used to load data in their data warehouse overnight and take decision based on BI, next day. Today organizations need actionable insights faster than ever before to stay competitive, reduce risks, meet customer expectations, and capitalize on time-sensitive opportunities – Real-time, near real-time.
Real-time is often defined in microseconds, milliseconds, or seconds, while near real-time in seconds, minutes.
With real-time analytics, the main goal is to solve problems quickly as they happen, or even better, before they happen. Real-time recommendations create a hyper-personal shopping experience for each and every customer.
The  Internet of Things (IoT) is revolutionizing real-time analytics. Now, with sensor devices and the data streams they generate, companies have more insight into their assets than ever before.
Several industries are using this streaming data & putting real-time analytics. 
·         Churn prediction in Telecom
·        Intelligent traffic management in  smart cities
·        Real-time surveillance analytics to reduce crime
·        Impact of weather and other external factors on stock markets to take trading decisions
·        Real-time staff optimization in Hospitals based on patients 
·        Energy generation and distribution based on smart grids
·         Credit scoring and  fraud detection in financial & medical sector
Here are some real world examples of real-time analytics:
·        City of Chicago collects data from 911 calls, bus & train locations, 311 complaint calls & tweets to create a real-time geospatial map to cut crimes and respond to emergencies
·        The New York Times pays attention to their reader behavior using real-time analytics so they know what’s being read at any time. This helps them decide which position a story is placed and for how long it’s placed there
·        Telefonica the largest telecommunications company in Spain can now make split-second recommendations to television viewers and can create audience segments for new campaigns in real-time
·        Invoca, the call intelligence company, is embedding IBM Watson cognitive computing technology into its Voice Marketing Cloud to help marketers analyze and act on voice data in real-time.
·        Verizon now enables artificial intelligence and machine learning, predicting the customer intent by mining unstructured data and correlations
·        Ferrari, Honda & Red Bull use data generated by over 100 sensors in their Formula 
One cars and apply real-time analytics, giving drivers and their crews the information they need to make better decisions about pit stops, tire pressures, speed adjustments and fuel efficiency.
Real-Time analytics helps getting the right products in front of the people looking for them, or offering the right promotions to the people most likely to buy. For gaming companies, it helps in understanding which types of individuals are playing which game, and crafting an individualized approach to reach them.
As the pace of data generation and the value of analytics accelerate, real-time analytics is the top most choice to ride on this tsunami of information.
More and more tools such as Cloudera Impala, AWS, Spark, Storm, offer the possibility of real-time processing of Big Data and provide analytics,

Now is the time to move beyond just collecting, storing & managing the data to take rapid actions on the continuous streaming data – Real-Time!! 

Read more…

Digital Transformation in Utilities sector

Posted by Sandeep raut on December 11, 2016 at 4:30 in OSes
It is easy to take for granted the technology we have at our disposal. We flick a switch and the lights go on, we turn on the tap and clean water comes out. We don’t have to worry about gas for cooking. 
But today the Utilities industry is under pressure to simultaneously reduce costs and improve operational performance.
Utilities sector is a bit late in  digital innovations than  Retail,  Banking or  Insurance. With energy getting on the digital bandwagon with online customer engagement, smart sensors and better use of  analytics, Utilities are now adopting it.
Digital technology gives utility companies the opportunity to collect much richer, customer level data, analyze it for service improvements, and add new services to change the way customers buy their products.
Smart technology will be used to monitor home energy usage, to trigger alerts when previously established maximum limits are being reached, and to offer ‘time of use’ tariffs that reward consumers for shifting demand from peak times. 
Electricity is the most versatile and widely used form of energy and global demand is growing continuously.  Smart grids manage the electricity demand in sustainable, reliable and economic manner.
Advantages of Digital Transformation:
  • Digital makes customer self-service easy.
  • Digitally engaged customers trust their utilities.
  • Customer care, provided through digital technology, offers utilities both cost-to-serve efficiencies and improved customer intimacy.
  • Digital technology brings the capability to provide more accurate billing and payment processing, as well as faster response times for changing addresses and bills, removing and adding services, and many other functions
  • Using Mobile as a primary customer engagement channel for tips and alerts
  • Predictive maintenance with outage maps and real time alerts to service engineer helps reduce the downtime and costs
  • Smart meters allows utilities organizations to inform their customers about the energy consumption, tailor products and services to their customers while   achieving significant operational efficiencies at the same time

Meridian, a New Zealand energy company, launched PowerShop, an online energy retail market place that gives customers choice and control over how much power they buy and use. This helped Meridian attract online consumers and extend its reach of core retail offering.
Google’s Nest, an  IoT enabled energy efficiency management gives details about consumption patterns and better control.
Thames Water, UK’s largest provider of water uses digital for remote asset monitoring to anticipate equipment failures and respond in near real time.
Big Data analytics and actionable intelligence gives competitive advantage by gained efficiency. 
IBM Watson with its  cognitive computing power helped utilities identify trend and pattern analysis, predict which assets or pieces of equipment are most likely to cause points of failure. 
Today more than ever, utilities companies are asking: “How can we be competitive in this digital world?” People, whether they are customers, citizens or employees, increasingly expect a simple, fast and seamless experience. 
Read more…

Product recommendations in Digital Age

Posted by Sandeep raut on November 27, 2016 at 22:56 in OSes
By 1994 the web has come to our doors bringing the power of online world at our doorsteps. Suddenly there was a way to buy things directly and efficiently online.
Then came eBay and Amazon in 1995....... Amazon started as bookstore and eBay as marketplace for sale of goods.
Since then, as  Digital tsunami flooded, there are tons of websites selling everything on web but these two are still going great because of their product recommendations.
We as customers, love that personal touch and feeling special, whether it’s being greeted by name when we walk into the store, a shop owner remembering our birthday, helping us personally to bays where products are kept, or being able to customize a website to our needs. It can make us feel like we are single most important customer. But in an online world, there is no Bob or Sandra to guide you through the product you may like. This is where recommendation engines do a fantastic job.
With personalized product recommendations, you can suggest highly relevant products to your customers at multiple touch points of the shopping process. Intuitive recommendations will make every customer feel like your shop was created just for them.
Product recommendation engines can be implemented by collaborative filtering, content-
based filtering, or with the use of hybrid recommender systems.
There are various types of product recommendations:
           ·        Customers who bought this also bought - like Amazon
           ·        Best sellers in store – like HomeDepot
           ·        Latest products or arriving soon – like GAP
           ·        Items usually bought together – like Amazon
           ·        Recently views based on history – like Asos
           ·        Also buy at checkout – like Lego
There are many benefits that a product recommendation engine can do for  digital marketing and it can go a long way in making your customers love your website and making it their favorite eCommerce site to shop for.
Advantages of product recommendations:
·        Increased conversion rate
·        Increased order value due to cross-sell
·        Better customer loyalty
·        Increased customer  retention rates
·        Improved  customer experience
Application of  Data Science to analyze the behavior of customers to make predictions about what future customers will like.  Big Data along with  machine learning and  artificial intelligence are the key to product recommendations.
Understanding the shopper’s behavior on different  channels is also a must in personalizing the experience. Physical retail, mobile, desktop and e-mails are the main sources of information for the personalization engines
Amazon was the first player in eCommerce to invest heavily on product recommendations. Its recommendation system is based on a number of simple elements: what a user has bought in the past, which items they have in their virtual shopping cart, items they’ve rated and liked, and what other customers have viewed and purchased. Amazon has used this algorithm to customize the browsing experience & pull returning customers. This has increased their sale by over 30%.
Yahoo, Netflix, Yahoo, YouTube, Tripadvisor, and Spotify are other famous sites taking advantage of the recommender systems. Netflix ran a famous 1 million dollars competition from 2006 till 2009 to improve their recommendation engine.
Many commercial product recommendation engines are available today such as Monetate, SoftCube, Barilliance, Strands etc.
Ultimately most important goal for any eCommerce platform is to convert visitors into paying customers. Today the customer segmentation era as gone and its  hyper- personalization. 
Product recommendations are extremely important in digital age !!
Read more…

Using Data Science for Predictive Maintenance

Posted by Sandeep raut on October 15, 2016 at 5:23 in OSes
Remember few years ago there were two recall announcements from National Highway Traffic Safety Administration for GM & Tesla – both related to problems that could cause fires. These caused tons of money to resolve.
Aerospace, Rail industry, Equipment manufacturers and Auto makers often face this challenge of ensuring maximum availability of critical assembly line systems, keeping those assets in good working order, while simultaneously minimizing the cost of maintenance and time based or count based repairs.
Identification of root causes of faults and failures must also happen without the need for a lab or testing. As more vehicles/industrial equipment and assembly robots begin to communicate their current status to a central server, detection of faults becomes more easy and practical.
Early identification of these potential issues helps organizations deploy maintenance team more cost effectively and maximize parts/equipment up-time. All the critical factors that help to predict failure, may be deeply buried in structured data like equipment year, make, model, warranty details etc and unstructured data covering millions of log entries, sensor data, error messages, odometer reading, speed, engine temperature, engine torque, acceleration and repair & maintenance reports.
Predictive maintenance, a technique to predict when an in-service machine will fail so that maintenance can be planned in advance, encompasses failure prediction, failure diagnosis, failure type classification, and recommendation of maintenance actions after failure.
Business benefits of Data Science with predictive maintenance:
  • Minimize maintenance costs - Don’t waste money through over-cautious time bound maintenance. Only repair equipment when repairs are actually needed.
  • Reduce unplanned downtime - Implement predictive maintenance to predict future equipment malfunctioning and failures and minimize the risk for unplanned disasters putting your business at risk.
  • Root cause analysis - Find causes for equipment malfunctions and work with suppliers to switch-off reasons for high failure rates. Increase return on your assets.
  • Efficient labor planning — no time wasted replacing/fixing equipment that doesn’t need it
  • Avoid warranty cost for failure recovery – thousands of recalls in case of automakers while production loss in assembly line

TrainItalia has invested 50M euros in Internet of Things project which expects to cut maintenance costs by up to 130M euros to increase train availability and customer satisfaction.

Rolls Royce is teaming up with Microsoft for Azure cloud based streaming analytics for predicting engine failures and ensuring right maintenance.
Sudden machine failures can ruin the reputation of a business resulting in potential contract penalties, and lost revenue. Data Science can help in real time and before time to save all this trouble.
Read more…

Customer 360º view in Digital age

Posted by Sandeep raut on October 3, 2016 at 3:44 in OSes
In today’s  digital age of customer  hyper-personalization, organizations identify opportunities for real time engagement based on data-driven understanding of customer behavior.
Customers have taken control of their purchase process. With websites, blogs, Facebook updates, online reviews and more, they use multiple sources of information to make decisions and often engage with a brand dozens of times between inspiration and purchase.
It’s important that organizations collect every customer interaction in order to identify sentiments of happy & unhappy customers.
Companies can get a complete 360º view of customers by aggregating data from the various touch points that a customer may use, to contact a company to purchase products and receive service/support.
This Customer 360º snapshot should include:
  • Identity: name, location, gender, age and other demographic data
  • Relationships: their influence, connections, associations with others
  • Current activity: orders, complaints, deliveries, returns
  • History: contacts, campaigns, processes, cases across all lines of business and channels
  • Value: which products or services they are associated with, including history
  • Flags: prompts to give context, e.g. churn propensity, up-sell options, fraud risk, mood of last interactions, complaint record, frequency of contact
  • Actions: expected, likely or essential steps based on who they are and the fact they are calling now

The 360º view of customers, also often requires a  big data analytics strategy to marry structured data (data that can reside in the rows and columns of a database), with unstructured data (data like audio files, video files, social media data). 
Many companies like Nestle, Toyota are using social media listening tools to gather what customers are saying on sites like Facebook and Twitter, predictive analytics tools to determine what customers may research or purchase next.
What are the returns of Customer 360º:
  • All customer touch point data in a single repository for fast queries
  • Next best actions or recommendations for customers
  • All key metrics in a single location for business users to know and advise customers
  • Intuitive and customizable dashboards for quick insights
  • Real time hyper personalized customer interaction
  • Enhanced customer loyalty

Customer 360º helps achieve Single View of Customer across Channels – online, stores, marketplaces, Devices – wearables, mobile, tablets, laptops & Interactions – purchase, posts, likes, feedback, service.

This is further used for customer analytics – predict  churn, retention, next best action, cross-sell & up-sell opportunities, profitability, life time value.
Global leaders in customer experience are Apple, Disney, Emirates.
A word of caution though - Focus & collect only that customer data, which can help to improve the  customer journey.
Read more…

Why Data Science Is The Top Job In Digital Transformation

Posted by Sandeep raut on September 26, 2016 at 6:58 in OSes
Digital Transformation has become a burning question for all the businesses and the foundation to ride on the wave is being data driven.
DJ Patil & Thomas Davenport mentioned in 2012 HBR article, that Data Scientist is the sexiest job of the century, and how true!  Even the latest Glassdoor ranked Data Scientist at 1 st in top 25 best jobs in America.
Over the last decade there’s been a massive explosion in both the data generated and retained by companies. Uber, Airbnb, Netflix, Wallmart, Amazon, LinkedIn, Twitter all process tons of data every minute and use that for revenue growth, cost reductions and increase in customer satisfaction.
Most industries such as Retail, Banking, Travel, Financial Sector, Healthcare, and Manufacturing want to be able to make better decisions. With speed of change and profitability pressures on the businesses, the ability to take decisions had gone down to real time. Data has become an asset for every company, hence they need someone who can comb through these data sets and apply their logic and use tools to find some patterns and provide insights for future.
Think about Facebook, Twitter and other  social media platforms, smartphone apps, in-store purchase behavior data, online website analytics, and now all connected devices with  internet of things are generating tsunami of new data streams.
All this data is useless if not analyzed for actions or new insights.
The importance of Data Scientists has rose to top due to two key issues:
  • Increased need & desire among businesses to gain greater value from their data
  • Over 80% of data/information that businesses generate and collect is unstructured or semi-structured data that need special treatment 

Data Scientists:

  • Typically requires mix of skills - mathematics, statistics, computer science, machine learning and most importantly business knowledge
  • They need to employ the R or Python programming language to clean and remove irrelevant data
  • Create algorithms to solve the business problems
  • Finally effectively communicate the findings to management

Any company, in any industry, that crunches large volumes of numbers, possesses lots of operational and customer data, or can benefit from social media streams, credit data, consumer research or third-party data sets can benefit from having a data scientist or a data science team.

Top data scientists in the world today are:
  • Kirk D Borne of BoozAllen
  • D J Patil Chief Data Scientist at White House
  • Gregory Piatetsky of kdnuggets
  • Vincent Granville of Analyticsbridge
  • Jonathan Goldman of LinkedIn
  • Ronald Van Loon

Data science will involve all the aspects of statistics, machine leaning, and artificial intelligence, deep learning & cognitive computing with addition of storage from big data.

Read more…

Data Analytics alone cannot deliver effective automation solutions for the industrial IOT

Posted by David Oro on July 15, 2016 at 15:00 in OSes

Article: A Brief History of Field Programmable Devices (FPGAs)-data-analytics-alone-cannot-deliver-effective-automation-solutions-industrial-iot-min-jpg

By Akeel Al-Attar. This article first appeared here. 

Automated analytics (which can also be referred to as machine learning, deep learning etc.) are currently attracting the lion’s share of interest from investors, consultants, journalists and executives looking at technologies that can deliver the business opportunities being afforded by the Internet of Things. The reason for this surge in interest is that the IOT generates huge volumes of data from which analytics can discover patterns, anomalies and insights which can then be used to automate, improve and control business operations.


One of the main attractions of automated analytics appears to be the perception that it represents an automated process that is able to learn automatically from data without the need to do any programming of rules. Furthermore, it is perceived that the IOT will allow organisations to apply analytics to data being generated by any physical asset or business process and thereafter being able to use automated analytics to monitor asset performance, detect anomalies and generate problem resolution / trouble-shooting advice; all without any programming of rules!

In reality, automated analytics is a powerful technology for turning data into actionable insight / knowledge and thereby represents a key enabling technology for automation in Industrial IOT. However, automated analytics alone cannot deliver complete solutions for the following reasons:

i- In order for analytics to learn effectively it needs data that spans the spectrum of normal, sub normal and anomalous asset/process behaviour. Such data can become available relatively quickly in a scenario where there are tens or hundreds of thousands of similar assets (central heating boilers, mobile phones etc.). However, this is not the case for more complex equipment / plants / processes where the volume of available faults or anomalous behaviour data is simply not large enough to facilitate effective analytics learning/modelling. As a result any generated automated analytics will be very restricted in its scope and will generate a large number of anomalies representing operating conditions that do not exist in the data.

ii- By focussing on data analytics alone we are ignoring the most important asset of any organisation; namely the expertise of its people in how to operate plants / processes. This expertise covers condition / risk assessment, planning, configuration, diagnostics, trouble-shooting and other skills that can involve decision making tasks. Automating ‘Decision making’ and applying it to streaming real-time IOT data offers huge business benefits and is very complementary to automated analytics in that it addresses the very areas in point 1 above where data coverage is incomplete, but human expertise exists.

Capturing expertise into an automated decision making system does require the programming of rules and decisions but that need not be a lengthy or cumbersome in a modern rules/decision automation technology such as Xpertrule. Decision making tasks can be represented in a graphical way that a subject matter expert can easily author and maintain without the involvement of a programmer. This can be done using graphical and easy to edit decision flows, decision trees, decision tables and rules. From my experience in using this approach, a substantial decision making task of tens of decision trees can be captured and deployed within a few weeks.

Given the complementary nature of automated analytics and automated decisions, I would recommend the use of symbolic learning data analytics techniques. Symbolic analytics generate rules/tree structures from data which are interpretable and understandable to the domain experts. Whilst rules/tree analytics models are marginally less accurate than deep learning or other ‘blackbox models’, the transparency of symbolic data models offer a number of advantages:

i- The analytics models can be validated by the domain experts
ii- The domain experts can add additional decision knowledge to the analytics models
iii- The transparency of the data models gives the experts insights into the root causes of problems and highlights opportunities for performance improvement.

Combining automated knowledge from data analytics with automated decisions from domain experts can deliver a paradigm shift in the way organisations use IOT to manage their assets / processes. It allows organisations to deploy their best practice expertise 24/7 real time throughout the organisation and rapidly turn newly acquired data into new and improved knowledge.

Below are example decision and analytics knowledge from an industrial IOT solution that we developed for a major manufacturer of powder processing mills. The solution monitors the performance of the mills to diagnose problems and to detect anomalous behaviour:

The Fault diagnosis tree below is part of the knowledge captured from the subject matter experts within the company

Article: A Brief History of Field Programmable Devices (FPGAs)-fault-diagnosis-tree-min-jpg



The tree below is generated by automated data analytics and relates the output particle size to other process parameters and environmental variables. The tree is one of many analytics models used to monitor anomalous behaviour of the process.

Article: A Brief History of Field Programmable Devices (FPGAs)-automated-data-analytics-min-jpg



The above example demonstrates both the complementary nature of rules and analytics automation and the interpretability of symbolic analytics. In my next posting I will cover the subject of the rapid capture of decision making expertise using decision structuring and the induction of decision trees from decision examples provided by subject matter experts.

Read more…

Sponsor

Workshops

Webinars