Vaingast S. Beginning Python Visualization: Crafting Visual Transformation Scripts

Издательство: Apress
Количество страниц: 363
Язык: English
Год издания: 2009Книга издательства Apress - Beginning Python Visualization: Crafting Visual Transformation scripts, которая посвящена визуализации данных с помощью Python и смежных инструментов. Уникальная в своём роде работа, т. к. на Питоне такие вещи делают довольно редко. Автор книги говорит, что сам искал руководство для своей работы, и поскольку не нашёл, ему пришлось разбираться самостоятельно.В книге приводится несколько примеров, как можно визуализировать данные. Например, с помощью Python можно извлечь данные из GPS-ресивера через серийный порт и уже на компьютере обработать как угодно, в том числе создавать графики и диаграммы. Автор показывает всё это на примере конкретного GPS-приёмника и модуля PySerial.Самое главное в книге — подробная демонстрация, как работать с известными для визуализации математических и научных данных MatPlotLib, NumPy и SciPy, а также с библиотекой PIL (Python Imaging Library) для простейшей обработки изображений.Navigating the World of Data Visualization
The Environment
Python for Programmers
Data Organization
Processing Text Files
Graphs and Plots
Math Games
Science and Visualization
Image Processing
Advanced File Processing
Additional Source ListingI was always drawn to math and computers, ever since I was a kid playing computer games
on my Sinclair ZX81
When I attended university, I had a special interest in numerical analysis,
a field that I felt combines math and computers ideally. During my career, I learned of
MatLAB, widely popular for digital signal processing, numerical analysis, and feedback and
control. MatLAB’s strong suits include a high-level programming language, excellent graphing
capabilities, and numerous packages from almost every imaginable engineering field. But
I found that MatLAB wasn’t enough. I worked with very large files and needed the ability to
manipulate both text and data. So I combined Perl, AWK, and Bash scripts to write programs
that automate data analysis and visualization. And along the way, I’ve developed practices and
ideas involving the organization of data—for example, ways to ensure file names are unique
and self-explanatory.
With the increasing popularity of the Internet, I learned of GNU/Linux and the open
source movement. I made an effort to use open source software whenever possible, and so
I’ve learned of GNU-Octave and gnuplot, which together provide excellent scientific computing
functionality. That fit well on my Linux machine: Bash scripts, Perl and AWK, GNU-Octave and
gnuplot.
Knowing I was interested in programming languages and open source software, a friend
suggested I give Python a try. My first impression was that it’s just another programming language:
I can do most anything I need with Perl and Bash, resorting to C/C++ if things got hairy.
And I’d still need GNU-Octave and gnuplot, so what’s to gain? Eventually, I did learn Python
and discovered that it is far better than my collection of tools. Python provides something that
is extremely appealing: it’s a one-stop shop—you can do it all in Python.
I’ve shared my enthusiasm with friends and colleagues. Many who expressed interest with
the ideas of data processing and visualization would ask, Can you recommend a book that
teaches the ideas you’re preaching? And I would tell them, Of course, numerous books cover
this subject! But they didn’t want numerous books, just one, with information distilled to
focus on data analysis and visualization. I realized there wasn’t such a title, and this was how
the idea for this book originated.Navigating the World of Data Visualization
Gathering Data
Case Study: GPS Data
Scanning Serial Ports
Recording GPS Data
Data Organization
File Format
File Naming Conventions
Data Location
Data Analysis
Walking Directories
Reading CSV Files
Analyzing GPS Data
Extracting GPS Data
Data Visualization
GPS Location Plot
Annotating the Graph
Velocity Plot
Subplots
Text
Tying It All Together
Final Notes and ReferencesThe Environment
Operating Systems
GNU/Linux
Windows
Choosing an Operating System
Then Again, Why Choose? Using Several Operating Systems
The Python Environment
Versions
Python
Python Integrated Development Environments
Scientific Computing
Plotting
Image Processing
Additional Python Packages
Installation Summary
Additional Applications
Editors
A Short List of Text Editors
Spreadsheets
Word Processors
Image Viewers
Version Control Systems
Licensing
Final Notes and ReferencesPython for Programmers
What Is Python?
Interactive Python
Invoking Python
Entering Commands
The Interactive Help System
Moving Around
Running Scripts
Data Types
Numbers
Strings
Booleans
Data Structures
Lists
Tuples
Dictionaries
Sets
Variables
Statements
Printing
User Input
Comments
Flow Control
Some Built-in Functions
Defining Functions
Generators
Generator Expressions
Object-Oriented Programming
Modules and Packages
The import Statement
Modules Installed in a System
The dir Statement
Final Notes and ReferencesData Organization
File Name Conventions
Date and Time in a File Name
Useful File Name Titles
File Name Extensions
In Conclusion
Other Schemes
File Formats
CSV File Format
Binary Files
Readme Files
INI Files
XML
Other File Formats
Locating Data Files
Organization into Directories
Searching for Files
Indexing
Catalogs
Files vs. a Database
Final Notes and ReferencesProcessing Text Files
Text and Strings
Splitting Text
Joining Strings
Converting Strings to Numbers
Find and Replace
Stripping Strings
String Formatting
String Conditionals
More on Strings
Files
Opening a File
Closing a File
Writing Text
Reading Text
Working with Text Files
Example: Character, Word, and Line Count
Example: head and tail
Example: Splitting and Combining Files
Example: Searching Inside a Text File
Example: Working with Comments
Example: Extracting Numbers from a Text File
CSV Files
The CSV Module
The CSV.reader Object
The CSV.writer Object
More CSV Functionality
DictReader and DictWriter Objects
Date and Time
Time Module
The struct_time Tuple
Parsing and Formatting Date and Time
The Epoch: Linearizing the Time Base
Additional Time and Date Functions
Regular Expressions
Regular Expression Patterns
Special Sequences
Alternatives
Ranges
When to Use Regular Expressions
Internationalization and Localization
Locale
Unicode Strings
Final Notes and ReferencesGraphs and Plots
The Matplotlib Package
Interactive Graphs vs. Image Files
Interactive Graphs
Saving Graphs to Files
Plotting Graphs
Lines and Markers
Plotting Several Graphs on One Figure
Line Widths and Marker Sizes
Colors
Controlling the Graph
Axis
Grid and Ticks
Subplots
Erasing the Graph
Adding TextAxis Labels and Legend
Text Rendering
Mathematical Symbols and Expressions
More Graph Types
Bar Charts
Histograms
Pie Charts
Logarithmic Plots
Polar Plots
Stem Plots
Additional Graphs
Getting and Setting Values
Setting Figure and Axis Parameters
Patches
Example: Adding Arrows to a Graph
Example: Some Other Patches
Final Notes and ReferencesMath Games
Modules math and cmath
Example: A Newton Fractal
Module random
Using random to Solve Probability Questions
Random Sequences
Module NumPy
Array Creation
Slicing, Indexing, and Reshaping
N-Dimensional Arrays
Math Functions
Array Methods and Properties
Other Useful Array Functions
Final Notes and ReferencesScience and Visualization
Finding Your Way: Variables and Functions
SciPy
Linear Algebra
Solving a System of Linear Equations
Vector and Matrix Operations
Matrix Decomposition
Additional Linear Algebra Functionality
Numerical Integration
More Integration Methods
Interpolation and Curve Fitting
Piecewise Linear Interpolation
Polynomials
Uses of Polynomials
Spline Interpolation
Solving Nonlinear Equations
Special Functions
Signal Processing
Functions where, select, and find
Functions diff and split
Waveforms
Fourier Transform
Example: FFT of a Sampled Cosine Wave
Window Functions
Filtering
Filter Design
Example: Heart-Rate Monitor
Example: Moving Average
Final Notes and ReferencesImage Processing
Reading, Writing, and Displaying Images
Reading Images from File
Image Attributes
Displaying Images
Converting File Formats
Image Manipulation
Creating New Images
Copy and Paste
Crop and Resize
Rotate
Image Annotation
Annotating with Geometrical Shapes
Text Annotations
Image Processing
Matrix Representation and Colors
Example: Counting Objects (Five Parts)
Image Arithmetic
Image Filtering
Final Notes and ReferencesAdvanced File Processing
Binary Files and Random Access
Example: Reading the Nth Field
Example: Efficient Tail Implementation
Example: Creating a Fixed-Size File
Example: Recording Time-Based Binary Data
Object Serialization
The Pickle Module
Command-Line Parameters
argv
Example: Creating a Fixed-Size File (Stand-Alone Script)
OptParse Module
The FileInput Module
File and Directory Manipulation
Module glob
Additional os Module Functionality
Additional os.path Module Functionality
Module shutil
File Compression
Example: A Compressed tar File
Comparing Files
Module filecmp
Module difflib
Final Notes and References
APPENDIX
Additional Source Listing
Nudge Subplots
Magic Square Arrows
Fractal Function Source Code