[PDF] [PDF] Advanced Python for Scien0fic Compu0ng

[PDF] Expert Python ProgrammingPdf

language itself but rather on the tools and techniques used to work with it In other words this book describes how an advanced Python developer works

[PDF] Intermediate/Advanced Python

Topics › Review of basic data structures › Accessing and working with objects in python › Numpy – How python actually stores data in memory – Why numpy 

[PDF] Advanced Programming with Pythonpdf - CERN Indico

Advanced Programming with Python DISCLAIMER: The presented material relies heavily on Python Advance course carried out at CERN The material is also 

[PDF] Advanced Python for Scien0fic Compu0ng

Advanced Python for Scien0fic Enthought Python Distribukon / Canopy provides scienkfic and math Display in window or output to PDF, SVG, PNG, etc

[PDF] A Python Book: Beginning Python, Advanced Python, and Python

15 déc 2013 · 2 1аааIntroductionаннаPython 201анна(Slightly) Advanced Python Topics PDFаннаhttp://www davekuhlman /python_book_01 pdf

[PDF] Mastering Python Programming (with Advanced Topics)

Web Development Scripting Series Python Programming Training Course: TTPS4820 Mastering Python Programming (with Advanced Topics) Duration: 5 

[PDF] Advanced Python Programming

17 juil 2000 · Python programmers who want to know more Programmers who aren't afraid of gory details Disclaimer This tutorial is aimed at an advanced 

[PDF] Safari OReilly Books Online - Home

Learn the core features of Python as well as advanced topics such as regular expressions, multithreaded programming, Web/Internet and network development, 

[PDF] advanced python programming pdf

[PDF] advanced python programming pdf 2018

[PDF] advanced python projects pdf

[PDF] advanced python scripting for arcgis pro

[PDF] advanced r packages

[PDF] advanced real analysis book pdf

[PDF] advanced real analysis textbook pdf

[PDF] advanced regular expression in python

[PDF] advanced research methodology pdf

[PDF] advanced search app for android

[PDF] advanced segmentation adobe analytics

[PDF] advanced setting app for android

[PDF] advanced shell scripting pdf

[PDF] advanced spoken english lessons pdf

[PDF] advanced sql book reddit

AdvancedPythonforScien2ficCompu2ng

MichaelMilligan

milligan@msi.umn.eduFollowalong!

Togetthemostoutofthis...

• BasicknowledgeofPython • WorkingPythoninstall(feelfreetouseours!) - EnthoughtPythonDistribuCon/Canopyprovides scienCficandmathlibrariespre-installed • MSIlogin+SSHorNX• Followalong!

WhyPythonforScienCficCompuCng?

• Rapiddevelopment - Easy,readablesyntax- VersaCletoolsforexperimentaCon/learning- Comprehensivelibraries • PowerfulFeatures - Processdataatnear"naCvecode"speeds- ExcellentvisualizaConpackages- Comprehensivelibraries

Whenyouleavetoday,youshouldbeableto...

• PrograminteracCvelywithipython

• understandthebasicsofnumpyandscipy• Efficientlycomputewithlargearraysofdata• Loadandsavedatato/fromfilesondisk• Usematplotlibtoplotdata• TakeadvantageofsupercompuCngresourceswith

parallelcompu2ng • Knowwheretoturnformorehelpwiththesetopics

Details

• WearedescribingEnthoughtPythonDistribuCon. - EssenCally:Pre-assembledcompilaConofPython2.7+ numpy,scipy,otherusefullibraries - Freeforacademicuse,abasicversionisfreefornon- commercialuse - Yourcomputers,departments,etcmayhaveadifferent • InMSI:moduleloadpython-epd

WorkshopConvenCons

• UNIXshellcommandsareindicatedwiththepercentsign. • IPythoninterpreter commandshaveIn/Outlabels • Neithersignindicates

IPython:InteracCvePython

• PowerfulenvironmentforinteracCvework • Runas"ipython"from anyterminal • --pylabopConauto- loadsnumpy,setsupgraphicsforploang • Inspectanyobjectwith "?"orhelp()

IPython:InteracCvePython

• BuildupaworkspaceofobjectsandfuncCons • Fullhistoryaccess throughOut[],%recall,up/downarrowkeys • %load,%edit,or%run externalfiles • Lotsmore,type%magic

NumPyandSciPy

• NumPyprovides:

• thebasicarrayandmatrixdatatypes• EfficientimplementaConsoflow-levelmathoperaCons• Alargelibraryofhigh-levelmathfuncConsbuiltfrom

efficientprimiCves • SciPyprovides: • Ahomeforawidevarietyofopen-sourcemathemaCcal andscienCficalgorithms • ModulesforopCmizaCon,signalprocessing,linear algebra,staCsCcs,interpolaCon,andmore

NumPyarrays

• ArraydatatypewithvectorizedoperaCons(similartoMatlaborIDL) • SupportssameoperaConsas

Pythonlisttype

• ...excepteveryelementisof samedatatype • ...sotheycanbestoredin memorypackedlikeCarrays

NumPyarraysarefast

Here we are comparing a "pure Python" loop to the equivalent in numpy

MulCdimensionalarrays

• NumPyarraysarerectanglesinarbitrarilymanydimensions • +-*/operate element-by-elementforsame-shapearrays

Arrayslicing

• IndexnotaCongivesaccesstoany"slice"ofanarray • Arrayslicescanbe assigned-thischangestheoriginalarray • X=M[1,:,:].copy() wouldavoidchangingM

Othercommonmethods

• Numpyarrayshavemanyusefulbuilt-inmethods

Othercommonmethods

• ...andthenumpymoduleprovidesmore

CondiConsandtests

• Vectorizedlogicaloperators+indexingfuncCons • Outputofindex funcConscanbeusedtoslicearrays

CondiConsandtests

• Vectorizedlogicaloperators+indexingfuncCons • Outputofindex funcConscanbeusedtoslicearrays

Moreusefulnumpymodules

• numpy.fft-FFTs,forward/inverse,1-DandN-D • numpy.random-generaterandomnumbers,many distribuConstochoosefrom • numpy.matrix-specialarraysthatobeymatrixmath• numpy.polynomial-moduleforrepresenCngand manipulaCngarbitrarypolynomials

Ploangmadeeasy

• Matplotlibprovideshigh-quality2-D(andsome3-D)ploang • DisplayinwindoworoutputtoPDF,SVG,PNG,etc• Implementedasmodularobject-orientedsystem • PylabprovidesaMatlab-ishinteracCve interfacetoMatplotlib • Accesswithipython--pylab• Defaultstopoppingupplotsinaseparatewindow

Somebasicexamples...

Someadvancedexamples...

• Theseexamplesarefromthematplotlib.orgexamplessecCon...

Someadvancedexamples...

Someadvancedexamples...

Someadvancedexamples...

Someadvancedexamples...

SciPyexpandsthemenu

• Clusteringalgorithms(scipy.cluster)

• IntegraConandODEs(scipy.integrate)• InterpolaCon(scipy.interpolate)• Inputandoutput(scipy.io)• Linearalgebra(scipy.linalg)• MulC-dimensionalimageprocessing(scipy.ndimage)• OpCmizaConandrootfinding(scipy.opCmize)• Signalprocessing(scipy.signal)• Sparsematrices(scipy.sparse)• SpaCalalgorithmsanddatastructures(scipy.spaCal)• SpecialfuncCons(scipy.special)• StaCsCcalfuncCons(scipy.stats)• Andthensome...

SciPyisalsofast

• MostSciPyrouCnesusefastNumPylow-levelmathoperaCons • SomeSciPyrouCnesusehighlyopCmized externallibraries - E.g.scipy.linalglinkstoBLAS,LAPACKorMKL behindthescenes

Dataondisk

• Chancesareyouwanttoloadandsavedata • numpyandscipy.ioofferavarietyoffaciliCes

Dataondisk:textfiles

• Verycommonforsmallerdatasets:simplecolumnsofnumbers • numpy.loadtxt()-simpleinterface,gooddefaults• numpy.genfromtxt()-morecomplex,handles

Dataondisk:textfiles

• Numpy.savetxt()-writetocolumnsofnumbers

Dataondisk:binaryformats

• Binarydataismuchmorescalable

• Smallerfilesondisk• Fastertoloadandsave• Maybenecessarytoexchangedatawithothersopware• SCcktoportable(machine-independent)formats

Dataondisk:binaryformats

• NumPyna2veformat(.npy)

• numpy.load()andnumpy.save()• Orusenumpy.savez()tostoremanyarraysincompressed.npz• Fast,portable,butmostlyonlysupportedbyPython

• scipy.io.matlab-supportforMatlab(.mat) • scipy.io.loadmat()andscipy.io.savemat() • scipy.io.idl-read(nosave)IDL.savfiles • scipy.io.readsav()

Dataondisk:binaryformats

• Manystandardformatssupported

• scipy.io.netcdf-NetCDF3interface• h5pyexposesHDF5API• PyTablesisanexcellenthigh-levelinterfacetoHDF5• pyfitsforFITSdatasets• Etc...

ScalingupwithparallelizaCon

• Forbigjobsyouwilleventuallywanttoparallelizeyourcode • ThePythoninterpreterhastroublewith mulCthreading-mulC-processisusuallybest • Approachdependsontheproblemyouneed tosolve

Parallelprocesses

• Manyjobsneedtoprocesslotsofdata,don'tneedtocommunicateamongstthemselves • SomeCmescalled"embarrassinglyparallel"• GNUParallel--asimplewaytolaunchjobs • Launchonejobforeveryfileinadir,lineinafile,etc• CanworkwithPBSonitascatousemanynodes

GNUParallelexample

GNUParallelexample

• -jshouldmatchppn(unlessyouknowwhatyou'redoing)-thisisprocessespernode • Willrunonejobperlineofinputonstdinorin argfile-maxofnodes*ppnrunningatonce • See"manparallel"formorefeatures

MPIforPython

• MPI"MessagePassingInterface"enablesparallelprocessestocommunicateefficiently • Commonlyoneprocesswillbe"controller"and manageworkerprocesses

• Inherentsupportforscaser-gatheroperaCons• MPIiswell-supportedonourclusters• mpi4pyinterfacestoMPIfrominsidePython• CaveatforMPIgurus:numpydoesnothave

Examplewithmpi4py

• Simple"Helloworld"script

Examplewithmpi4py

• Simple"Helloworld"script

Morewithmpi4py

• Possibletopassnumpyarrayslikebuffers

Morewithmpi4py

• Alsoworkswith(pickle-able)Pythonobjects • MuchslowerthanC-basedarrays,butveryconvenient

Toomuchtocover...

• Ipythonnotebook-connecttoipythonwithabrowserforaMathemaCca-likenotebookinterface

• PyCUDAandPyOpenCL-GPUcompuCng• SymPy-MathemaCca-stylesymbolicmath• DatabasesareeasytoconnecttoPython;or

IPythonnotebookexample

• Example:IPythonnotebookwithpylabandsympy • notebookcreates graphicalloginabrowser • sympy:symbolicCAS• Totrythis:

CommunityandDocumentaCon

• AcCvelydevelopedandsupported

• ExcellentdocumentaCon• www.python.org/doc• Scipy.org• wiki.scipy.org• Ipython.org• matplotlib.org• Mpi4py.scipy.org

NextStep

• Hands-on • Youcanalsoruntheexamplesonyour laptop'sPythondistribuCon • Enthoughtisinstalledinalllabsandon supercomputersatMSI • FullacademicversionofEnthoughtCanopy installed(notdefaultyet) • QuesCons!quotesdbs_dbs20.pdfusesText_26