PythonForDataScientist

XAI( Explainable AI  ) is grabbing lime-light in machine learning. How can we be sure that image classification algo is learning faces not background ? Customer wants to know why loan is disapproved? Globally important variable might not be responsible/ imp for individual prediction. Here XAI comes to rescue- We have taken data from  classification_data This has some sensor values and an output class.  A) Data Explainability - what are the basic understanding required from data perspective.  1)       Identify missing values, co-linear feature, feature interaction, zero importance feature, low important feature, single value feature and handle missing values, remove/ handle features accordingly. 2)       Missing values- no missing values from data description 3)       No good correlation between variables- can be seen from correlation plots 4)    ...

autoreload IPython extension to reload modules before executing user code. autoreload reloads modules automatically before entering the execution of code typed at the IPython prompt. % load_ext autoreload % autoreload 2 from foo import some_function some_function () Out[4]: 42 # open foo.py in an editor and change some_function to return 43 some_function () Out[6]: 43 https://ipython.org/ipython-doc/3/config/extensions/autoreload.html

This is simple program to convert audio into text. I have used speec_recognition library to do it. import speech_recognition as sr r = sr.Recognizer() with sr.Microphone() as source:     print("Speak Anything :")     audio = r.listen(source)     try:         text = r.recognize_google(audio)         print("You said : {}".format(text))     except:         print("Sorry could not recognize what you said") Output- Speak Anything : You said : internet is required to access the Google API if there is no internet you will not be able to convert voice into text to install SpeechRecognition- https://pypi.org/project/SpeechRecognition/ one of the dependencies is PyAudio which is not supported after 3.6. For python verison 3.6+ one needs to download wheel (  https://python101.pythonlibrary.org/chapter39_wheels.html )  and install PyAudio separately. https://sta...

Python library Geopandas used for plotting geo-spatial data- import geopandas world = geopandas.read_file(geopandas.datasets.get_path('naturalearth_lowres')) world.plot() <matplotlib.axes._subplots.AxesSubplot at 0x2fd60a9ecf8>

python date datatype- datetime pandas date datatype- Timestamp 1) creating pandas datframe with string dates value ## datetime date_data= pd.DataFrame(np.random.randint(10,100, (4,3)), columns=['A','B','C']) dates= ['2 june 2013', '5 Aug 2015', '2015-07-09', '7/12/2014'] date_data.index= dates A B C 2 june 2013 52 61 89 5 Aug 2015 21 69 89 2015-07-09 88 23 13 7/12/2014 43 39 21 creating string into pandas datetime format- date_data.index= pd.to_datetime(date_data.index) date_data A B C 2013-06-02 59 93 77 2015-08-05 33 15 28 2015-07-09 63 19 25 2014-07-12 29 36 92 2) Time difference in pandas- ...