The module B8IS100 Data Management and Analytics has proven exceptionally fulfilling in all aspects covered. Learning about data management, analytic methods, the related tools and applying this knowledge in a practical sense has provided a swift learning curve. The journey has been a truly fascinating exercise and has given me an appetite to explore further learnings and qualifications in this hugely active and growing field.
We have learned that Information Systems (IS) comprise a functioning network of hardware, software and networks that people and organisations use to collect, create, analyse, secure and distribute data.
IS Hardware will comprise all the physical user devices, data processing systems, media storage devices along with all physical materials associated with and required for information processing.
IS Software will include both the applications software (programs, tools etc) along with the Operational Support System (OSS). OSS is the network entity through which the system personnel (not end users) conduct all system maintenance, upgrades, GUI modifications and more. Basically anything which is outside the domain of the system end users/clients.
IS Networks provide the systems interconnectivity across the global telecommunications network system. These networks comprise The Internet, Intranets and Extranets. They provide the essential interconnection necessities for all successful modern day businesses and enterprises to maintain, grow and protect their products, services and resources.
The People aspect of an IS includes the end-users who utilise the system and the IS specialists and Business Analysts who design, develop, implement, maintain, upgrade and secure the IS entities.
There are four major types of Information Systems: –
- MIS – Management Information Systems
- DSS – Decision Support Systems
- ESS – Executive Support Systems
- TPS – Transactional Support Systems
The following diagram, sourced at Paginas MIS illustrates and details the above four specified IS systems types: –
We also looked at the decision making scenario’s presented by each Information System, the organisational layer these decisions will generally occur at, plus the type of decision input/output that can be anticipated. We came to understand how structured decision making generally occurs at Operations Level, semi-structured tactical decision making usually at mid-management level plus how long term strategic decision making will occur at the most senior levels of management within a company right up to CEO level.
The following diagram sourced at Process Consultant Blog Spot summarises typical roles associated with each of the three above specified decision layers: –
It is appropriate to include a subsection on IS security since the volumes of data being generated and transgressing networks and corporations continues to soar at never before anticipated growth rates.
The scale and pace with which organisations are having to implement information systems presents ever new security challenges. Data managers and their systems analysts must ensure their Management Information Systems have robust protective measures in place to protect their business interests, their clients and staff’s personal information and the overall investments.
I found the following article summing up perfectly the challenges and exposures that modern businesses must contend with and deal with to secure all aspects and resources of their business from on-line interference, damage, theft and other cyber security breaches.
Dealing with today’s Information Systems complexity
Information systems complexity is the enemy of security. From mobile to the cloud and practically everything in between, all businesses have information systems complexities which are creating big security issues. This complexity rears its ugly head time and again in businesses both small and large and, given our dependence on information, appears to be on an exponential growth track.
Information systems complexity isn’t just about the quantity of systems on the network. It goes much deeper than that and includes factors like:
- Multitude of applications, virtual machines, and even cloud service providers that are both known and unknown (i.e. other people in the business doing their own thing without involving IT and security staff)
- Guidelines, standards, and policies that some (rarely all) people may or may not be held accountable to
There’s another element of complexity – often the biggest – that can create immeasurable security risks in your environment at any given time: your users. The human aspects of computer usage such as thinking and decision-making have a profound impact on IT management and information risks.
It’s not just our own networks that are complex either. The very threats we’re fighting off can be very complicated as well. The techniques used by criminal hackers and advanced malware are beyond the comprehension of many people, including IT professionals. Further complicating matters is the reality that it’s hard to protect against something that hasn’t yet happened. [END OF REPOST]
Above quoted items helps highlight the exceptional focus, urgency and prioritisation that all organisations and the system designers must place on ensuring rigorous and impenetrable security firewalls are placed around all access points to Information Systems. Due to the exponential rise that will occur over forthcoming years with The Internet of Things and the over twenty billion predicted connected devices, all aspects of security are being tested from each and every angle and by a global plethora of on-line hackers and dedicated web cyber-criminal communities. I found the content of the above published article extremely informative regarding the sheer volume of management information systems that criss-cross our lives on a daily basis. It is an article that highlights the responsibility that system providers and administrators must adhere to in their diligence with the build and management of each and every Information System.
The following concise definition of business intelligence, is courtesy of Webopedia: –
Business intelligence (BI) represents the tools and systems that play a key role in the strategic planning process within a corporation. These BI systems allow a company to gather, store, access and analyze corporate data to aid in decision-making. Generally these systems will illustrate business intelligence in the areas of customer profiling, customer support, market research, market segmentation, product profitability, statistical analysis, and inventory and distribution analysis to name a few.
To supplement the excellent material our class received from out lecturers uploads to Moodle, my on-line research led me to the following overview summary diagram of BI toolsets. I will use this diagram courtesy of Predictive Analytics Today to briefly expand on the toolsets that comprise the current BI environment.
- Spreadsheets – predominantly MS Office Excel and MAC Numbers
- Reporting and Querying – Organisations primarily using their own software to report, query, sort, filter and display data
- OLAP – Online Analytical Tools to enable users perform interactive analyse on data from multiple sources incorporating multidimensional view.
- Digital dashboards – Real-time user interfaces that allow graphical presentation of current operational status and ability to quickly extract historical reports for all previous reports too
- Data mining – Identifying patterns in large data sets encompassing various methods e.g. artificial intelligence, machine learning, statistics, database systems
- Data warehousing – Centralised storage location for gathered data retrieved from multiple sources. Serves as repository for all data that will have future predictive analysis conducted on. Robust back up processes and infrastructure are key requirements of all industry best practice data warehouses
- Process mining – Analysis based on historical events logs available in an information system and utilised for process mining
- Business performance management – Processes for managing the performance of a business or organisational unit
- GIS information systems – Supports and facilitates geographic information system reporting
In a 2001, Gartner analyst Doug Laney defined the 3V’s of Big Data, namely volume, velocity and variety.
Some years later, following further studies and evaluations, IBM introduced a fourth V to Big Data, Veracity.
Volume – Data is being generated in exponentially increasing volumes each day. Sources include on-line and mobile phone transactions, social media platforms, industry computer logs. The Big Data strategists and tools developers are continually evaluating how business and mankind ensures that all the data being generated and collected globally on a daily basis is being utilised correctly to benefit organisations and society, Apache Hadoop is the industry recognised tool that is used to perform data analytics on the vast swathes of unstructured data that is being gathered and exchanged across the Internet every second of every day.
Velocity –Velocity not only refers to the speed that data is being generated and collected but also to the turn around time on the utilisation of the gathered data into useful and meaningful information for industry and society. The advent of smartphones and tablets has placed new demands on Big Data storage systems and how velocity factors are responding to data traffic.
Variety – Variety refers to the multiple formats that data is now being presented across the web and then channeled into data warehouses. It is primarily unstructured data i.e. e-mails, photos, on-line purchase transactions, social media exchanges, tweets, phone call records to name just a handful.
Here is an IBM comment that helps put Gartners Big Data 3V’s into context (IBM Big Data Hub): “On Facebook alone we send 10 billion messages per day, click the like button 4.5 billion times and upload 350 million new pictures each and every day. If we take all the data generated in the world between the beginning of time and the year 2000, it is the same amount we now generate every minute! This increasingly makes data sets too large to store and analyze using traditional database technology. With big data technology we can now store and use these data sets with the help of distributed systems, where parts of the data is stored in different locations, connected by networks and brought together by software”.
3 V’s of Big Data diagram courtesy of Data Science Central Blog
Veracity – IBM state on their Big Data Analytics Hub: The average billion dollar company is losing $130 million a year due to poor data management. Veracity refers to the uncertainty surrounding data, which is due to data inconsistency and incompleteness, which leads to another challenge, keeping big data organized.
- Completeness [for example: Mandatory fields vs Optional fields compliance and correctness with the supplied and stored data]
- Timeliness [for example: Publishing data when agreed and obliged to do so, Up to Date information from Customers Services for incoming inquiries, accurate cross checking on credit card accounts]
- Consistency [For example: All credit card data info across an enterprise is fully synched up, close of dates on promotions are validated across the system, eligibility for offers is up to date with customer purchasing activity]
- Validity [For example: no invalid characters appearing in a data string and thus rendering it useless or missing field]
- Integrity [For example: All business rules pertaining to primary and foreign key attributes are correctly adhered to]
- Accuracy [For example: Adherence to a format that date of birth must be captured in]
CONCLUDING: PROJECTS ASSESSMENTS
The projects undertaken for module B8IS100 Data Management and Analytics have been extremely interesting, informative, stimulating and at times challenging too.
Getting acquainted with Fusion Tables highlighted to us the typical tools that are freely available to all who wish to engage with and learn more regarding data analytics. We worked with raw data from government publications (e.g. census population figures, district crime rate figures), formatted the data within excel to ensure that it was structured using pivot table requirements and then went about manipulating the data using Google Fusion Tables (from Google Drive) to generate a heat map. Creating a heat map for population densities in Ireland with census figure from 2011 was our particular project assignment in this case.
Following our lectures on R programming language and completing the R:School on-line course, we downloaded R:Studio to our laptops our PC’s. Again, we would work with data that was captured in excel. In this case we saved our data in comma separated value format (.csv) and used the .csv file in our R:Studio workspace. Working with R and R:Studio has been a hugely beneficial exercise that illustrated and equipped us with the basics on working with large volumes of data and converting this into charts for interpretation and decision making. Again, a big task to try and do with excel became a repeatable and manageable exercise in R.
Our journey with projects kicked off in Semester One when we studied SQL, standard query language. We learned how to create our own relational database, load up our data and to practically recreate a real living user test case. Our exercise required us to create a relational data base for a video rental store and to run queries on customer accounts including their rental history and status.
All in all, I am very pleased with the projects we were assigned. The preparation our lecturer provided us with regarding theory, practical aspects and support was superb. When combining all the learnings across this module and others from the overall course, I have to say that I have surprised myself with how quickly I came to grips with so much of the material and the appetite the course has given me to expand now upon the learnings and groundwork done.
Data Management and Big Data Analytics is a compelling subject matter to delve into. It will no doubt provide countless opportunities for students who study this course at the college to pursue rewarding careers. It will also inspire a good many to take up further studies and become highly qualified specialists within data management and data analytics.