Rajiv : Thanks Sharon for offering your help, once again. 

Sharon : No worries Rajiv. Based on your mail, it seems pretty evident that Excel has reached its limits and has started to show its downsides. Nonetheless, its still a go-to tool for data analysis. The problem is scale and complexity which after a limit, Excel fails to address

Rajiv : Yes Sharon. I felt so. There are just too many things to take care of when dealing with large volume as well as variety of datasets

Sharon : That's natural to happen for a company which is on a growth path. The organization can no longer depend solely on Excel for Data Analysis. Moreover, it can't even rely on spreadsheets to store its data. It's just too risky and error-prone to keep doing so

Rajiv : What's your suggestion then? Shall we dump Excel and move on to something else?

Sharon : Yes Rajiv. At least in case like yours. If data size is smaller and the audience is non-technical, nothing beats Excel. However, you've crossed that stage and have become closer to the data team than business. You need to change your tool of choice and think about Databases!

Rajiv : Databases? Yes, I've heard of them. I once did a SELECT * FROM TABLE inside a MySQL database. Don't know more than that. 

Sharon : What you wrote was a SQL statement which is a language used to query data from a database. 

Don't worry. Databases would be the next natural progression for you towards your journey to become a better Data Analyst and very soon SQL would be your tool of choice for data analysis. But first check out the hint below to get answer to the question : 

How do Databases/SQL help tackle the challenges discussed above?

Rajiv : I'm getting confused a bit between Databases and SQL. You are using these terms interchangeably. 

Sharon : Sorry about that. Database can be imagined like an Excel workbook which stores data in the form of sheets. While, SQL offers mechanism to query, process and wrangle data (just like select, filter, sort, pivot operations in Excel)

Rajiv : Got it. So, does this mean I can do the same operations I did in Excel? If yes, then how does SQL actually help make my job easier or faster?

Sharon : Firstly, it helps you convert the logic in your mind to solutions much faster than Excel. Rather than remembering so many menu options and clicking them in a slow sequence to get your results, it offers english like simple constructs to arrive at your solutions. Check out the video below to get a quick comparison of performing data operations in Excel vs doing the same using SQL in a database

Rajiv : Wow! That's a huge time saver. On top of it, it's plain english. I write what I think. Logic building looks so simple here :)

Sharon : Without doubt. This is one of the strongest reason why SQL being invented in the 70s still exists and is getting more and more popular by each passing day. 

Rajiv : I can't wait to lay my hands on SQL and speed up my work!

Sharon : Hold on. For that you'll need the data stored within a database and access to the same. It would be a good idea to understand what's a relational database (the one used to store data in tabular format) before you get started with SQL. I promise I'll keep it to minimal introduction and focus on harnessing SQL a lot more!

Rajiv : Most welcome. I'm ready to learn

Sharon : All right. Check out the hint below to get a quick idea about what relational databases are

Rajiv : That was a quickie! Looking forward to learn more about them

Sharon : For sure. As we progress, we'll keeping returning back to learn more concepts about relational databases. However, you mentioned in your mail that GlobalMart off late is also receiving lot of JSON like hierarchical data. 

You know relational databases these days support storage of semi-structured data also? This means you can work seamlessly with both structured (tabular) as well as semi-structured (document, JSON,XML) data within a single relational database?

However, in modern times, the types of data and need for storage has diversified so much that relational databases aren't the only kinds existing. There are purpose built databases which exist for different needs. I'd like you to just have a quick glance over the below set of videos to get an overall idea

Rajiv : Sure Sharon. Happy to learn

Sharon : Great! check out the hint below to get a short introduction to document databases which were purpose built to store hierarchical data

Sharon : The below hint introduces Graph databases. They are the ones used to implement social media apps like Facebook or Instagram

Sharon : Finally, a new database format primarily used in Data Warehouses called columnar databases. They have revolutionized how Data Warehouses power data driven decision making at organizations. Would be nice to have a quick idea about them

Sharon : Hope this opens up your mind about databases and their importance in the world of Data Analytics

Rajiv : Thanks Sharon. It really opens up a pandora's box. I hope I delve deeper into them in future

Sharon : For sure. However, the good news is, 80-85% of the world out there still runs on relational databases. This means, knowing SQL, you can solve up to 80% of your clients' analytics needs.

Rajiv : This means I should dive into SQL right away!

Sharon : Indeed. This is one skill which will get you exponential returns in your career. Let me mail you some problems which will help you pick up SQL skills fast enough. 

Rajiv : Looking forward!