SQL For Data Science (Coursera)

By

Overview

SQL is one of the most basic skills for developers, DBAs, and all the data team (analysts, engineers, and scientists), but constantly it is underestimated. In general, people are reluctant to keep different languages in the same project to avoid issues with professionals, and sometimes that forces the use of ORMs when the fastest option would be to run some stored procedures. No problem here looking by the perspective of management: if all engineers could use the same technology, there are less problems with work allocation and situations of absence (vacations for a small number of people who master a key technology means they can't all leave at the same time).

Growth of cloud services and introduction of new tools to query massive amounts of data have made systems designs that delegate a large portion of business logic to the databases and SQL based tools popular again. Concepts like "data lakes" and "data warehouses" are starting to be applied in smaller companies (not only big techs) and those are dependent on good performance. 

So, to put in a nutshell, SQL is growing again as a necessary tool to design modern systems - specially in data engineering. 

This is the description of the course obtained in Coursera:




Pros

This course has an environment where we can run the SQL queries and get results. This is interesting because a big obstacle to overcome when learning SQL is installing and configuring the database (unless you are using MySQL, of course =). Let's look for an example at a typical Postges installation: you install the database, must start it - which if you are windows will probably run as a service - and then you have to use a tool such as pgAdmin to connect to the server. But pgAdmin is actually a web app which runs locally and you have to use it in your browser. That is a long path before you can write your first "SELECT"s, right?

Cons

One thing that appeared a little out of the course scope is the graded project. First, I have the impression it is not mandatory for graduation, so why have it there in the first place? Second, some part of it already expect you to know a few Data Science concepts. I mean, the title has a "for Data Science" in it, but all the exercises until the final project are about SQL only. This transition could be a little more gradual.

Conclusion

This is a great course, specially to grasp basic SQL concepts. The course instructor has great communication skills and provide good practical examples. Also, the practical exams are awesome to really digest all the theory in the lessons. 

Comments

Post a Comment

Popular posts from this blog

Dealing With Large Features in Git Repos

By
Image
There is not a "correct" way to organize your servers and git repos and each company has its own problem and solutions. Some companies use test servers to run some very basic verifications in their codebase before deploying new features to production. Some even have a staging server that is as similar to production as possible, where new features are tested for a last time before deploying to prod. Those servers usually run deployed code that is in the development, staging, and master/main branches in the git repo. This would be what the usual flow of development looks like: Another good practice many companies adopt is the versioning of the code to keep track of what is deployed on each server. There is a lot about versioning alone that should be discussed in other posts, but let us adopt the format major.minor.build for our example. Dealing with Bugs in Tests Now, let us imagine that a bug was discovered in one of your features when it was being tested in the Test Server. W
Show me more »

Distributed Computing with Spark SQL (Coursera)

By
Image
Overview This course was a great positive surprise regarding the way the practical exams worked. You can think of this as a great introduction to Databricks, along with Spark, and a fact I think few people know: you can use Databricks for free if you create a community account. Spark is one of the most important tools (if not the most important) for large data processing (a.k.a. Big Data). The main advantage it provides over traditional data processing tools is the possibility of horizontal scaling, which makes scalability almost limitless. Spark is a basic tool for Data Engineers and one of the largest problems one can face when learning Data Engineering is the problems we are trying to solve are usually for big corporations with a huge pile of data, which often involves high costs to simulate such situations. This course is part of the SQL Basics for Data Science specialization. Here are the course details from Coursera: Pros As mentioned before, the best part of this course was the
Show me more »

Foreign Visitors in Brazil - 2005 to 2015 - Part II

By
Image
So this is the continuation of our previous analysis of foreign visitors from 2005 to 2015. The first thing we see is how the variable of interest (number of arrivals) behaves through the years. It is observable that there has been a slight increase in the number of foreign visitors since 2013. It is also possible to see how the accumulated visits are distributed through the continents on the graph below. From this graph we can understand that this increase is specially because of South America. For the other two continents with most visitors, Europe and North America, there has been a slight decrease in arrivals. Below we have the graph of arrivals per continent in this 10 year window. South America and Europe have by far the larger number of visitors. Next we observe how the sum of arrivals is ditributed through the months of the year. As expected, there is a larger number of arrivals during summer time in Brazil. However, there is an unexplained increase in July. So far, one possib
Show me more »