1、Mastering Python Scientific ComputingA complete guide for Python programmers to master scientific computing using Python APIs and toolsHemant Kumar MehtaBIRMINGHAM - MUMBAIMastering Python Scientific ComputingCopyright 2015 Packt PublishingAll rights reserved. No part of this book may be reproduced,

2、 stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.Every effort has been made in the preparation of this book to ensure the accuracy of the i

3、nformation presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book

4、.Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.First published: September 2015Production reference: 118091

5、5Published by Packt Publishing Ltd. Livery Place35 Livery Street Birmingham B3 2PB, UK.ISBN 978-1-78328-882-3CreditsAuthorHemant Kumar MehtaProject CoordinatorSanjeet RaoReviewersAusten GroenerSachin R. JoglekarProofreaderSafis EditingIndexerTejal SoniCommissioning EditorKartikey Pan

6、deyGraphicsJason MonteiroAcquisition EditorKevin ColacoProduction CoordinatorAparna BhagatContent Development EditorArshiya UmerCover WorkAparna BhagatTechnical EditorMohita VyasCopy EditorVikrant Phadke歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！About the AuthorHemant Kumar Mehta is a distr

7、ibuted and scientific computing enthusiast. He has more than 13 years of experience of teaching, research, and software development. He received his BSc (in computer science) Hons., master of computer applications degree, and PhD in computer science from Devi Ahilya University, Indore, India in 1998

8、, 2001, and 2011, respectively. He has experienceof working in diverse international environments as a software developer in MNCs. He is a post-doctorate fellow at an international university of high reputation.Hemant has published more than 20 highly cited research papers in reputed national and in

9、ternational conferences and journals sponsored by ACM, IEEE, and Springer. He is the author of Getting Started with Oracle Public Cloud, Packt Publishing. He is also the coauthor of a book named Internet and Web Technology, published by Kaushal Prakashan Mandir, Indore.He earned his PhD in the field

10、 of cloud computing and big data. Hemant is a member of ACM (Special Interest Group on High-performance Computing Education: SIGHPC-Edu), senior member of IEEE (the computer society, STC on cloud computing, and the big data technical committee), and a senior member of IACSIT, IAENG, and MIR Labs.I a

11、m extremely thankful to my PhD supervisors, namely Professor Priyesh Kanungo and the late Professor Manohar Chandwani from Devi Ahilya University. Their words work as continuous guiding lights in my career and life.I express heartfelt thanks to my dear student and friend, Pawan Pawar, for helping me

12、 develop some programs for this book.I am also thankful to the entire Packt Publishing team and the reviewers for their tremendous support in maintaining the highest quality of work in this book.Most of all, I thank my family. I am infinitely grateful to my parents. I thank my wife, Priya, and darli

13、ng sons, Luv and Darsh, for whom this acknowledgement cannot be covered in words.歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！About the ReviewersAusten Groener was raised in Southfield, Massachusetts, USA. He completed his BA in physics from Hartwick College and went on to pursue his MS and P

14、hD inphysics from Drexel University in Philadelphia, Pennsylvania, USA. He is a reputed astrophysicist, with research interests surrounding the detailed distribution of dark matter within the largest objects in the universegalaxy clusters. When he is not studying the cosmos, he enjoys spending his f

15、ree time developing software tools for other astronomers to use. Austen has a newfound interest in web development.I would like to thank my family and friends for their unwavering support. To my wife, Brittany: you are the love of my life, my best friend, and my inspiration.Sachin R. Joglekar is a c

16、omputer science graduate from BITS-Pilani (Goa campus) in India. His areas of interest primarily include machine learning and intelligent systems.He graduated in December 2014. Since then, he has been working as the cofounder of a start-up based in Mumbai. His work involves the design and developmen

17、t of server infrastructure and backend analytics for sensor networks. Sachin has also worked as an open source developer for SymPy, a symbolic computing library written in pure Python. His work at Google Summer of Code 2014 involved developing the vector module for SymPy.歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462

18、347，群里免費(fèi)提供500+本Python書籍！www.PacktPSupport files, eBooks, discount offers, and moreFor support files and downloads related to your book, please visit www.PacktP.Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the

19、 eBook version at www.PacktP and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.At www.PacktP, you can also read a collection of free technical articles, sign up for a range of free newsletters and

20、receive exclusive discounts and offers on Packt books and eBooks.TM/books/subscription/packtlibDo you need instant solutions to your IT questions? PacktLib is Packts online digital book library. Here, you can search, access, and read Packts entire library of books.Why subscri

21、be?Fully searchable across every book published by Packt Copy and paste, print, and bookmark contentOn demand and accessible via a web browserFree access for Packt account holdersIf you have an account with Packt at www.PacktP, you can use this to access PacktLib today and view 9 entirely free

22、 books. Simply use your login credentials for immediate access.歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！To my parents and my gurus, Late Prof. Manohar Chandwani and Prof. Priyesh Kanung

23、o歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！Table of ContentsixPrefaceChapter 1: The Landscape of Scientific Computing and Why Python?1Definition of scientific computingA simple flow of the scientific computation process Examples from scie

24、ntific/engineering domainsA strategy for solving complex problemsApproximation, errors, and associated concepts and termsError analysisConditioning, stability, and accuracy Backward and forward error analysis Is it okay to ignore these errors?Computer arithmetic and floating-point numbersThe backgro

25、und of the Python programming languageThe guiding principles of the Python language Why Python for scientific computing?Compact and readable code Holistic language design Free and open source Language interoperability Portable and extensible Hierarchical module systemGraphical user interface package

26、s Data structuresPythons testing framework Available librariesThe downsides of PythonSummary246781010111111121315151516161616161617171718 i 歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！Table of ContentsChapter 2: A Deeper Dive into Scientific Workflows andthe Ingredients of Scientific Computi

27、ng Recipes19Mathematical components of scientific computationsA system of linear equationsA system of nonlinear equations OptimizationInterpolation Extrapolation Numerical integrationNumerical differentiation Differential equationsThe initial value problem The boundary value problemRandom number gen

28、eratorPython scientific computingIntroduction to NumPy The SciPy libraryThe SciPy SubpackageData analysis using pandasA brief idea of interactive programming using IPythonIPython parallel computing IPython NotebookSymbolic computing using SymPyThe features of SymPy Why SymPy?The plotting librarySumm

29、aryChapter 3: Efficiently Fabricating and Managing Scientific Data2020212223232425252626272830303031313232353636374041The basic concepts of dataData storage software and toolkitsFilesStructured files Unstructured filesDatabasePossible operations on data Scientific data formatReady-to-use standard da

30、tasets Data generationSynthetic data generation (fabrication)Using Pythons built-in functions for random number generationBookkeeping functionsFunctions for integer random number generation4143434444454547505454555656 ii 歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！ Table of ContentsFunctions

31、 for sequencesStatistical-distribution-based functions Nondeterministic random number generatorDesigning and implementing random number generators based on statistical distributionsA program with simple logic to generate five-digit random numbersA brief note about large-scale datasets SummaryChapter

32、 4: Scientific Computing APIs for Python5657586062626465Numerical scientific computing in PythonThe NumPy package The ndarrays data structure File handlingSome sample NumPy programsThe SciPy package The optimization package The interpolation packageIntegration and differential equations in SciPy The

33、 stats moduleClustering package and spatial algorithms in SciPy Image processing in SciPySample SciPy programs Statistics using SciPy Optimization in SciPyImage processing using SciPySymbolic computations using SymPyComputer Algebra System Features of a general-purpose CAS A brief idea of SymPySymPy

34、 modulesSimple exemplary programs Basic symbol manipulation Expression expansion in SymPySimplification of an expression or formula Simple integrationsAPIs and toolkits for data analysis and visualizationData analysis and manipulation using pandasImportant data structures of pandas Special features

35、of pandasData visualization using matplotlibInteractive computing in Python using IPython Sample data analysis and visualization programsSummary65666666676970707070717171717273757676768081828282838383848485868689 iii 歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！Table of ContentsChapter 5: Per

36、forming Numerical Computing91The NumPy fundamental objectsThe ndarray objectThe attributes of an array Basic operations on arraysSpecial operations on arrays (shape change and conversion) Classes associated with arraysThe universal function objectAttributes MethodsVarious available ufuncThe NumPy ma

37、thematical modulesIntroduction to SciPy Mathematical functions in SciPy Advanced modules/packagesIntegrationSignal processing (scipy.signal) Fourier transforms (scipy.fftpack)Spatial data structures and algorithms (scipy.spatial) Optimization (scipy.optimize)Interpolation (erpolate) Linear

38、algebra (scipy.linalg)Sparse eigenvalue problems with ARPACK Statistics (scipy.stats)Multidimensional image processing (scipy.ndimage) ClusteringCurve fittingFile I/O (scipy.io)SummaryChapter 6: Applying Python for Symbolic Computing9292929394959797989899101101102102105106107109115117119119122123126

39、128129131Symbols, expressions, and basic arithmetic Equation solvingFunctions for rational numbers, exponentials, and logarithms PolynomialsTrigonometry and complex numbers Linear algebraCalculus VectorsThe physics module Hydrogen wave functions Matrices and Pauli algebraThe quantum harmonic oscilla

40、tor in 1-D and 3-D Second quantization132133134135135136138141142142142143143 iv 歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！ Table of ContentsHigh-energy Physics MechanicsPretty printingLaTeX PrintingThe cryptography module Parsing inputThe logic moduleThe geometry module Symbolic integrals

41、 Polynomial manipulation SetsThe simplify and collect operations SummaryChapter 7: Data Analysis and Visualization145146148149150151152154156158160161162163MatplotlibThe architecture of matplotlibThe scripting layer (pyplot) The artist layerThe backend layerGraphics with matplotlibOutput generationT

42、he pandas library Series DataFrame PanelThe common functionality among the data structures Time series and date functionsHandling missing dataI/O operationsWorking on CSV files Ready-to-eat datasetsThe pandas plottingIPythonThe IPython console and system shellThe operating system interface Nonblocki

43、ng plotting DebuggingIPython NotebookSummary164164164165165166168170170171172174181184186186190190191192194194195197198 v 歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！Table of ContentsChapter 8: Parallel and Large-scale Scientific Computing199Parallel computing using IPythonThe architecture o

44、f IPython parallel computingThe components of parallel computingThe IPython engine The IPython controllerIPython view and interfaces The IPython clientExample of performing parallel computingA parallel decorator IPythons magic functionsActivating specific views Engines and QtConsoleAdvanced features

45、 of IPython Fault-tolerant execution Dynamic load balancingPushing and pulling objects between clients and engines Database support for storing the requests and results Using MPI in IPythonManaging dependencies among tasksFunctional dependency Graph dependency Impossible dependenciesThe DAG dependen

46、cy and the NetworkX libraryUsing IPython on an Amazon EC2 cluster with StarClusterA note on security of IPythonWell-known parallel programming stylesIssues in parallel programming Parallel programming Concurrent programming Distributed programming Multiprocessing in Python Multithreading in PythonHa

47、doop-based MapReduce in Python Spark in PythonSummaryChapter 9: Revisiting Real-life Case Studies200201202202202203203203204205206207207207208209210213214214216217218220222222222224224225226227229231232233Scientific computing applications developed in PythonThe one Laptop per Child project used Pyth

48、on for their user interface ExpEYES eyes for scienceA weather prediction application in PythonAn aircraft conceptual designing tool and API in Python234234236239240 vi 歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！ Table of ContentsOpenQuake EngineSMS Siemag AG application for energy efficienc

49、yAutomated code generator for analysis of High-energy Physics data Python for computational chemistry applicationsPython for developing a Blind Audio Tactile Mapping SystemTAPTools for air traffic controlEnergy-efficient lights with an embedded systemScientific computing libraries developed in Pytho

50、nA maritime designing API by Tribon Molecular Modeling Toolkit Standard Python packagesSummaryChapter 10: Best Practices for Scientific Computing241243243245246247248249249250251252253The best practices for designingThe implementation of best practicesThe best practices for data management and appli

51、cation deploymentThe best practices to achieving high performance The best practices for data privacy and security Testing and maintenance best practicesGeneral Python best practices SummaryIndex254256258260261261262264265 vii 歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！歡迎加入非盈利Python編程學(xué)習(xí)交流QQ

52、群783462347，群里免費(fèi)提供500+本Python書籍！PrefaceI am absolutely convinced that in a few decades, historians of science will describe the period we are in right now as one of deep and significant transformations to the very structure of science. And in that process, the rise of free openly available tools play

53、s a central role.Fernando Perez, creator of IPythonThis book covers the Python APIs and toolkits used to perform scientific computing. It is highly recommended for readers who perform computerized engineering or scientific computations. Scientific computing is an interdisciplinary branch that requir

54、es a background in computer science, mathematics, general science (at least any one branch out of physics, chemistry, environmental science, biology, and others), and engineering. Python consists of a large number of packages, APIs, and toolkits for supporting the functionalities required by these d

55、iverse scientific and engineering domains.A large community of users, lots of help and documentation, a large collection of scientific libraries and environments, great performance, and good support make Python a great choice for scientific computing.What this book coversChapter 1, The Landscape of

56、Scientific Computing and Why Python?, introduces the basic concepts of scientific computing. It also discusses the background of Python, its guiding principle, and why using Python for scientific computing is efficient. ix 歡迎加入非盈利Python編程學(xué)習(xí)交流QQ群783462347，群里免費(fèi)提供500+本Python書籍！PrefaceChapter 2, A Deeper Dive into Sc