In today’s era of rapid technological development, programming languages are emerging and their hotness is constantly changing. Understanding the trend of popular programming languages is extremely important for programming learners to plan their learning paths and for industry practitioners to grasp the direction of technology. In this case, we will use Python to analyze the programming language data obtained from well-known technology websites, and gain insights into the mystery of their popularity.
We plan to extract programming language data from Stack Overflow, the world’s largest question-and-answer community for programmers, which publishes annual developer survey results that contain a wealth of information on programming language usage and are highly informative.
To get the data, we will use the requests library to send web requests and the BeautifulSoup library to parse the web content. Here is the code example to get the data
import requests from bs4 import BeautifulSoup send request get url https insights stackoverflow com survey response requests get url html content response text Use BeautifulSoup to parse the page soup BeautifulSoup html content html parser Here suppose we want to extract the programming language names and usage percentages According to the structure of the page, find the corresponding tags and class names to extract the data The following is the sample code, the actual situation may need to be adjusted according to the page update. adjust language names usage percentages for row in soup find all tr 1 skip header rows columns row find all td language names append columns 0 text strip usage percentages append float columns 1 text strip
The obtained data may have some incomplete or inconsistent formatting, which needs to be cleaned and organized. For example, there may be some null values or duplicate rows, which we use the pandas library to handle.
import pandas as pd Create DataFrame df pd DataFrame Language language names Usage Percentage usage percentages Remove duplicate rows df df drop duplicates Check if there are null values if df isnull any any If so, you can choose to remove the null rows or do something else df df dropna
Next, we analyze the collated data and present the results more intuitively through visualization. For example, we can draw bar graphs to compare the percentage of usage of different programming languages.
import matplotlib pyplot as plt Set Chinese fonts to solve Chinese display problems plt rcParams font sans serif SimHei plt rcParams axes unicode minus False Plot a bar graph plt bar df Language df Usage Percentage plt xlabel Programming language plt ylabel Usage percentage plt title Comparison of usage of popular programming languages plt xticks rotation 45 Rotate x-axis labels to prevent overlapping plt show
With the above code, we have successfully obtained programming language data from the web, cleaned and organized it, and plotted intuitive bar charts to show the popularity trend of different programming languages. Of course, this is just a simple example, you can further analyze the data, such as analyzing the trend by year, or combining it with other related data for a more comprehensive study.Python’s powerful data processing and analyzing capabilities provide unlimited possibilities for us to explore the secrets behind all kinds of data, whether it’s in the analysis of programming language trends or in many other fields, which is very useful. Python’s powerful data
Leave a Reply