SigmaWay Blog

In today’s world volume of data has been so large. This problem can be tackled through Hadoop-based Data Lake. A data lake is a storage repository that holds a vast amount of raw data in its native format until it is needed. There are some strengths, weakness, opportunity and treats.

Strengths:

A Hadoop-based data Lake is a low cost operation, as it is open source software and can be processed on low cost system. Hadoop has a capability of storing and processing all types of data, whether structured or unstructured, which incurs only a small proportion of cost of our currently traditional systems.

Weaknesses:

There can be a lot of confusion due to large volume, variety and velocity of big data. Although Hadoop-based Data Lake is open source and helped to meet the increasing demands of vendor’s products, it had not received complete success yet. There is a lot of security problem with this system. Open source community and vendors tried to eradicate such problem by supporting security and privacy requirement of the organization.

As the frequency of credit card fraud is increasing, so are the costs associated with these bad transactions that include, lost merchandise, lost profit, transportation costs, etc. Hence, enter fraud analysts, who are continuously updated through automated machine learning, about any sort of fraudulent activities, spotted by customers, online. Five ways to spot and prevent online fraud:
1. Users with no shopping history can be suspected, who may create new accounts only to test stolen credit card information.
2. Bulk orders of multiple costly items are to be suspected.
3. Hastening up a delivery process is to be suspected, as imposters try to get benefited as much as possible, before getting detected.
4. Multiple accounts or multiple shipping or billing addresses, with any one common factor, out of the three, should be cross-checked.
5. Accounts with multiple credit cards should also be cross-checked.
Read more at: http://www.business2community.com/big-data/5-telltale-signs-your-business-is-battling-credit-card-fraud-01271169

Big data like other day-to-day works, have also entered into fitness. Just like Fitbit and Nike+ Running App, some other technology tools are also available now. Swimmers can now use Speedo Aquabeat which is an underwater iPad. The tool Trace, used in surfing and wakeboarding can now be mounted to any board to track the motion, speed, height and the tricks used while landing. The AiQ Bioman Tank monitors temperature, heart rate, and breathing of our body. For volleyball and basketball players, Vert app measures the jump and send the data to the smartphone comparing ability over time. Prescription sunglasses used for outdoor sports are also at a high popularity now. Read more about this article at: https://channels.theinnovationenterprise.com/articles/7636-top-5-unique-sports-tech-gadgets-making-fitness-better

Big Data is a resource that helps organizations to understand and take control or corrective action to address problem such as customer centric service delivery, by accurately joining big data with Master Data Management (MDM). MDM helps to control the momentum of incoming data and tools that applied in MDM helps to verify facts about customers, vendors and products. Data definitions and rules previously established by the corporation are reflected in the MDM. Service Oriented Architecture (SOA), due to its agile nature, plays a key role in the corporation achieving its vision.
There is a symbiotic relationship between MDM and SOA. SOA provides the filtering needed for a steady stream of consistent data, which is an important aspect of data mining as the quality of data derived determines the accuracy of predictions.
Thus MDM is a valuable tool as it has the capacity to store and analyze large volumes of data to discover new patterns which leads to an increase in profitability.

For more information visit:
http://www.finextra.com/blogs/fullblog.aspx?blogid=11073

In today's world where manufacturing has turned into a truly global industry spanning over multiple time zones with multiple languages and cultures, language can prove to be a stumbling block for communication and hence success. The expression “a picture is worth a thousand words” has never had more practical meaning than today. Visuals can actually help us understand complex problems faster than just texts or numbers.
Let us look at what goes in the making of a great visual:
• Easy to Setup: In a learn environment where stress is imposed on minimizing waste it is of prime importance that the time and resources needed to setup, a visual is minimized.
• Easy to See: Even before we get down to the task of analyzing the graphs what must be kept in mind is that the graphs should be visible. And by visibility we also broadly mean that the visuals should be placed at strategically at the “right place”.
• Easy to Maintain: Information posted via a visual must be constantly updated and it would be more cost efficient if every consequent update does not render the previous version as unusable.
• Easy to Use: The visuals should be designed in such a way that everyone is able to use the information displayed- so that no special training is needed to read the information shown.
• Easy to Understand: Visual symbols can work great for communication only if the information displayed is readily usable for analysis. If further processing of data has to be done before they can be used, it greatly diminishes its functionality.

For more information visit:
http://www.apriso.com/blog/2015/07/the-5-es-great-lean-visuals-have-in-common/

A certain cross-country survey revealed that the Internet of Things (IoT) was first adopted by the Healthcare Industry. Organizations that adopted the IoT, believed that they could meet business challenges like patient engagement, loyalty, safety and satisfaction without any delay.
According to Laura DiDio, Director of IoT Strategies Enterprise, the need for IoT is felt highly by enterprises to address and solve pragmatic business issues. Andrew Brown, SA’s Executive Director of IoT and Mobility says that IoT can also be used for cost savings, development of new services, increased competitiveness and better understanding of the business.
It can be said that the Internet of Things has great potential in the field of healthcare and with evolvement of mobile capabilities its growth rate will be escalated.
Read more at: http://healthitanalytics.com/news/healthcare-industry-is-an-early-internet-of-things-adopter

Today, fast and accurate decisions are critical for an organization's success. But there is a risk of incorrect decisions if we rely on approximate sciences such as intuition and judgment of individual decision makers. The value of data can be realized only once a coherent ‘data strategy’ is established. A data strategy requires an organization to embed and integrate data analytics into the process of decision-making. The organization must seek and utilize data based insights that are most fruitful. Big Data demands a tighter integration of business functions and better mechanisms for integration. Evidently, various teams shall work together to understand and exploit cross-functional data. Identifying key data gaps and taking collective decisions for data gathering will ensure that specifically data with potentially useful information is collected. Read more at: http://www.bobsguide.com/guide/news/2015/Jul/13/big-data-small-data-and-fast-data-using-data-to-drive-better-decisions.html

 Through a randomized double-blind controlled trial, we can guarantee predictive analysis though these trials sometimes fail. So in real-world the chances of true predictive analytics is virtually none as randomized double-blind control trials are also highly suspected. Thus workforce analysts should start using correlational analytics by first accepting that predictive analytics probably doesn't exist in most organizational settings and that we need to do with correlational analytics. Then they can assess the risk of investing large sums based on analytics which are correlational rather than predictive. Next always run a small pilot or two before spending big money on implementing the results of "predictive analytics" as we now know it is just correlational analytics. Read more about it by Max Blumberg (Workforce analytics and automation consultant) at:  https://www.linkedin.com/pulse/dont-waste-money-predictive-analytics-doesnt-exist-hr-max-blumberg?trk=hp-feed-article-title-like

SME’s have a large number of advantages in using big data technology and it is found to play an important role for them recently. Firstly it is important to understand that what big data is. Big Data is defined as a popular term to describe exponential growth and availability of data, both structured and unstructured and is important for businesses. It is a series of combined data lakes used for more accurate analyses. SME’s should move to Big Data storage solutions because storing the new data generated each day is becoming expensive for the companies and its increasing their costs. When SME’s start using such combined datasets, they will have enough data to analyse and improve their decision making. Interest in big data has reached new heights for SME’s as they take their chances of capturing more information from everywhere possible. SME’s can also use big data to attract new customers, tap new markets and reduce costs. Some more ways how big data technology can help SME’s prosper are by using big data tools which increase their ability to use predictive analytics for decision making. It also helps SME’s move unused data into different less expensive places. Hence SME’s should definitely be looking forward to such big data technology for improved performance.

Read more at: http://www.smartdatacollective.com/bruce-robbins/329520/big-data-smes

The automobile industry can make use of data to improve vehicles, sustainability and profitability. Smart cars are the next generation of cars and data will assist in building these. Surveys have revealed that customers prefer cars with features that save time, ensure safety and sends warning alerts in case of malfunctions. In order to accomplish these needs, integration of technologies is required. Services based on car data can be introduced like sending a warning alert to replace car battery prior to its depletion. Smart infrastructure can be built by connecting roadside sensors, traffic signals and remote data centers to ensure a safe and smooth ride. Data can be analyzed to map travelling patterns by looking at how individuals behave in various road scenarios. Read more at: https://channels.theinnovationenterprise.com/articles/7719-what-smart-carmakers-can-learn-from-the-smart-phone-industry

Companies of every shape or size recently have moved to Big Data as they realized how beneficial it is for their organizations. To efficiently use the big data, companies need to collect and process it first, from multiple sources. Some data remains unused in the case of unstructured data mostly, which poses a problem later leading to clogging up data centres. Such unused data has been termed as – Dark Data which needs to be cleaned up or else it increases security risk and costs for the companies. Dark Data firstly needs to be identified. It is the information collected but not put to any specific use. Such data remains unused for years and takes up a lot of space which could have been used otherwise to store useful data. Dark Data remains unprotected for a while and poses risk of security breaches and mishaps. Dark Data is costly and dangerous but still companies keep them to make sure they don't lose any kind of records. Cleaning of Dark Data is time consuming but is important as well. One of the most effective ways of avoiding such data is by filtering the data which prevents data from piling up. Other ways to clean such data is by moving the Dark Data to different repositories using cloud computing. Dark Data can be deadly for organizations and hence it should be avoided before it poses any problem for the business.

 Read more at:  http://www.smartdatacollective.com/rick-delgado/323706/importance-cleaning-your-dark-data

User Behavior Analysis (UBA) helps in collection and analysis of huge amounts of user-specific data and access logs which can help us to identify unusual user behavior, investigating potential threats, detecting and responding to internal threats quickly, etc.
Enterprises can incorporate UBA in the existing security framework in the following ways:
1. Defining the problem – The most critical first step is to define the problem. Organizations that take time to identify their unique organizational security characteristics can determine the critical company assets and thus help UBA to safe guard their resources.
2. Identification – The next step would be to ascertain the group of users and the types of data that needs to be monitored.
3. Implementation – The third and the ultimate step is implementation. The various ways in which collected data can be used for use in security insights are:
• Visualization
• Drill-Down
• Aggregate
• Profile
UBA can be used by identifying the problem, gathering the relevant data, and establishing the required tools to gain visibility of the unseen threats in the network. Read more at: http://www.smartdatacollective.com/idan-tendler/326260/adopting-user-behavior-analytics-uba-solution

Data scientists are the ones who use sophisticated quantitative and computer science skills to both structured and analyse unstructured data as well as derive intuitions from the data and suggest actions. Data scientists can tackle with problems that are complex, huge in size and disorganized using several coding languages. To identify a data scientist following points are to be taken into consideration:

• Qualifications: Data scientists are required to have an advanced degree usually a masters or PhD in a quantitative discipline such as economics, statistics and their educational background may be diversified.

• Skills: Data scientists are efficient users of different tools used for analytics and are well versed with coding languages such as Python or Java used for writing programs, transformations etc. They are also have expert knowledge about statistical and machine learning models such as R and SAS.

• Dataset size: They usually work with datasets measured in gigabytes up to petabytes.

• Job responsibility: Data scientists are well equipped to work on every stage of analytics life cycle which also include data acquisition, transformation/cleaning, analytics to predict patterns of the datasets, prescribing actions and programming/automations to contribute to a firms data products.

In order to do away with bottlenecks in data science supply chains we need strategies by data scientists. Not only the productivity of the data scientists need to be improved, but also the increasing demands of data scientists. Technology solutions can be used in place of skilled professionals in this field which will present analytics to non-data scientists in familiar business terms. The models developed by the data scientists can be given to the business analysts then they can successfully get the work done without being technical.
The basic idea behind this is not to do away with data scientists but to keep them free for new challenges. Proper communication between data scientists and business analysts leads to a cultural shift which reduces the gap between these.
Read more at: http://www.forbes.com/sites/teradata/2015/07/08/sugar-spice-hadoop-and-everything-nice-a-recipe-for-more-data-scientists/

The term Big Data has become popular recently and is used to define exponential data development and availability, both structured and unstructured. Big data is crucial to businesses since more data implies precise analyses which results in efficient decision making, reduced risk and cost reduction. Big Data is used for more complex datasets and refers to application of predictive analytics and other advanced application tools to derive the results and seldom to the size of datasets. The three V’s of Big Data are namely: Volume, Velocity, and Variety. Apart from increasing varieties and velocities of data, data flows are very challenging. Variability, complexity are some other factors of Big Data. Data flows in from different sources and it’s difficult to connect them which has become a huge task these days. Big Data has become necessary for companies which helps them stay updated with the new technologies. Processing the Big Data smartly and accurately is worth a lot of income for the companies. Big Data demands amazing technologies to efficiently process such datasets in given deadlines. There were companies who spend a lot of money on data analytics and other management systems to increase their efficiency. Recently Big Data has increased the need for information organizations. Big Data also helps to grow and continue increasing the number of servers used which only adds to expenses of the company.

Read more at: http://www.smartdatacollective.com/anandsrivastava1988/328749/what-big-data-and-how-useful-it

It had been believed that Greece had struggled too much to meet the criteria for Euro entry. Official figures showing Greek deficit that were used to prove accession were found incorrect in 2004, and the authority continued to submit frequently revised statistics to the EU for the next decade. Data science could play a key role in protecting Greece. In 2011, Fact and Fiction in EU Governmental Economic Data (a research paper) used Benford’s Law up to a decade of economic data from 27 EU states. Benford’s Law states that in many naturally occurring collections of numbers the small digits occurs disproportionately often as leading significant digits. Benford’s Law was also used by financial auditors as a key to their fraud detection and the researchers tried to relate it to economic data. Researchers calculated the difference between the actual value and the expected value. After some research and analysis researchers found that Greece was the country with greatest deviation and they subject it to the effectiveness of Benford’s Law in determining the irregularities and manipulations in macroeconomic data. Now the question arises could data science and Benford’s Law let Greece to evade crises by keeping it away from Euro. The answer is no because the figures shown by Greece were hesitating. In fact it was Germany and France who broke the 3% budget deficit rule. Benford’s law might have helped, but the entry of Greece in Euro in fact was a political decision not economical. According to researchers, a little focus on Benford’s Law could help in recovering from such a devastating situation. Read more at: http://www.smartdatacollective.com/timoelliott/329137/could-data-science-have-saved-greece

Predictive analytics tools and techniques are becoming a rage nowadays because of their immense application in a variety of sectors. Collecting data and predicting future trends have been so easy. More and more business are vying for these tools as they help with predictive modelling of customer behavior. Even bigger potentials are lurking in the near future and organizations are investing in the development of these software keeping that in mind. Companies which have still not adopted these software are falling behind in competition with other firms. Thus there has been a surge in the usage of predictive analytics as every firm is craving for more information. Read more at: http://www.dataversity.net/how-your-business-can-benefit-from-predictive-analytics/

Self-service analytics softwares are being used by businesses now and it has given rise to a new need, data governance plans. These new softwares provide a host of benefits. These have made accessing and analyzing data a lot easier coupled with the ability to feed bad data back into enterprises. A competition has been brewing between software vendors regarding improved data governance. Companies are launching business intelligence products that stress data governance. Such softwares can be built around the company's pre-existing enterprise data infrastructure bringing multiple data sources into a common data warehouse accessible by the entire organization. Read more at: http://www.dataversity.net/adding-data-governance-tools-to-self-service-analytics/

A startling revelation has been made by a group of math and computer experts that has been hard to digest. Leaving behind data consumption giants like you tube, twitter in social media and astronomy, particle physics in science the field that has emerged as a leader is genomics.  Genomics generate the most electronic bytes per year relative to other fields. It has been a challenge to capture, store and interpret the genome encoded biological information. Data generation has increased at a fast pace with the growth of biological sciences. Leading edge data generation and processing technologies are needed to tackle the present situation be it in the field of biological sciences or social media or high tech science. To know more, please follow: http://www.dataversity.net/wheres-the-biggest-big-data-to-be-found-genomics/

The Energy industry is set to benefit largely from the usage of big data and analytics. Efficient use of information and improved analytics will help meet the difficult economic and environmental goals, bringing energy into the next century. For this we must update our energy infrastructure with the necessary software for collection and analysis of the required data. The potential of big data is such that it can help all customers support energy savings along with providing an environmental upside. Big data once employed correctly can improve operations for the benefit of businesses, customers and our environment. While the energy industry’s use of big data is future-focused, real progress is being made and these investments have improved operational efficiency and reduced the cost of energy all while helping protect the environment. Read more at: http://thehill.com/blogs/congress-blog/energy-environment/247559-big-data-means-big-energy-savings