Python Programming

[Mel81x+]

This pandemic and getting a vaccine has now turned me into a python pro haha. It's amazing what this language can do out of the box that so many others have to use extensive libraries for.

[nudge]

4 hours ago, Mel81x said:

This pandemic and getting a vaccine has now turned me into a python pro haha. It's amazing what this language can do out of the box that so many others have to use extensive libraries for.

A vaccine turned you into a python pro? 

 

 

[Mel81x+]

5 minutes ago, nudge said:

A vaccine turned you into a python pro? 

 

 

its what i had to do to get the vaccine that turned me into a python pro haha. And this is more apt for me.

[nudge]

12 hours ago, Mel81x said:

its what i had to do to get the vaccine that turned me into a python pro haha.

Tell me more...  

[MUFC]

Worry about you lot and your love for pythons.

[Mel81x+]

7 hours ago, nudge said:

Tell me more...  

Had to write a data script to call an API to pull data then feed it to another app that basically digested the data and some crunching which originally took 9 minutes but now happens in 17 seconds on average. It can be boosted so easily its ridiculous.

Also, they did other nefarious things but lets not go there.

[nudge]

@Mel81x care to take a look at something Python-related?

Basically, I needed to create multiple copies of a file (lets say, 100 copies of a file named myfile.xlsx) and then batch-rename the copies, using a list of provided new filenames. I did it in two steps:

1) creating multiple copies of a file: this was pretty straightforward, I simply used this code to do it:

import shutil

for i in range (100):
	shutil.copy2('myfile.xlsx', 'copy{}.xlsx'.format(i))
	

Gets the job done, quick and simple.

2) renaming the copies: for this, I wrote this .py script:

import os
import sys

with open(sys.argv[1]) as fh:
    for line in fh:
        sl = line.strip().split(",")
        old_name = sl[0]
        new_name = sl[1]
        os.rename(old_name, new_name)

then listed the old names of the copies and the new names in a text file like this:

copy1.xlsx,newname1.xlsx
copy2.xlsx,newname2.xlsx
copy3.xlsx,newname3.xlsx
etc
etc
etc

(It's obviously just an example; the actual new names are all unique (names of the company projects), but you get the idea.)

It all runs perfectly well and thus gets the job done  - my question is, do you have any ideas what would be the best way to join those two steps in a single script as to save even more time in the future? 

[Mel81x+]

10 hours ago, nudge said:

It all runs perfectly well and thus gets the job done  - my question is, do you have any ideas what would be the best way to join those two steps in a single script as to save even more time in the future? 

So to combine them and copy them without the rename and that just reduces the script. The copy step isn't really necessary as you have one file and you're just making copies of it so why even bother with renaming it, just copy it to its final destination.

import shutil
import os
import sys

with open(sys.argv[1]) as fh:
	for line in fh:
		sl = line.strip().split(",")
		file_tocopy = sl[0]
		new_file_name = sl[1]
		
		shutil.copy(file_tocopy, new_file_name)

Then the filelist kind of looks like this

myfile.xlsx, newname1.xlsx
myfile.xlsx, newname2.xlsx
myfile.xlsx, newname3.xlsx

P.S. I'd even go so far as to say variable naming is unnecssary and you can convert the entire copy statement like so

import shutil
import os
import sys

with open(sys.argv[1]) as fh:
	for line in fh:
		sl = line.strip().split(",")
		shutil.copy(sl[0], sl[1])

 

[nudge]

5 hours ago, Mel81x said:

So to combine them and copy them without the rename and that just reduces the script. The copy step isn't really necessary as you have one file and you're just making copies of it so why even bother with renaming it, just copy it to its final destination.

import shutil
import os
import sys

with open(sys.argv[1]) as fh:
	for line in fh:
		sl = line.strip().split(",")
		file_tocopy = sl[0]
		new_file_name = sl[1]
		
		shutil.copy(file_tocopy, new_file_name)

Then the filelist kind of looks like this

myfile.xlsx, newname1.xlsx
myfile.xlsx, newname2.xlsx
myfile.xlsx, newname3.xlsx

P.S. I'd even go so far as to say variable naming is unnecssary and you can convert the entire copy statement like so

import shutil
import os
import sys

with open(sys.argv[1]) as fh:
	for line in fh:
		sl = line.strip().split(",")
		shutil.copy(sl[0], sl[1])

 

Thank you, that works brilliantly!  

[Mel81x+]

1 minute ago, nudge said:

Thank you, that works brilliantly!  

Anytime. Glad you're now mass producing files and renaming them too. What else are you doing with python?

[nudge]

3 minutes ago, Mel81x said:

Anytime. Glad you're now mass producing files and renaming them too. What else are you doing with python?

Primarily web scraping for now, and then such small automating things like the copying/renaming every once in a while... Only started applying Python two months ago, didn't feel comfortable until then, but my confidence grows with every successful project  I'm really mindblown with what it can achieve, and how much new stuff I learn every time I read up on something new...

[Mel81x+]

2 hours ago, nudge said:

Primarily web scraping for now, and then such small automating things like the copying/renaming every once in a while... Only started applying Python two months ago, didn't feel comfortable until then, but my confidence grows with every successful project  I'm really mindblown with what it can achieve, and how much new stuff I learn every time I read up on something new...

Yeah it has lots of good productivity libraries for getting a lot of simple stuff done - number crunching, etc

[nudge]

55 minutes ago, Mel81x said:

Yeah it has lots of good productivity libraries for getting a lot of simple stuff done - number crunching, etc

Yeah, not to mention complete frameworks that tackle specific tasks even more productively and efficiently. I've been using Scrapy to build and run web spiders; I tried BeautifulSoup before and thought it was working just fine, but then tried Scrapy and since it's asynchronous, the speed at which it gets the work done is completely crazy... Granted, I find Scrapy's learning curve much steeper, and it usually takes me a long time to write the code I need, but it's still absolutely worth it. 

[Azeem]

Is Python better to start with if you are totally new to programming ? 

[Mel81x+]

14 hours ago, McAzeem said:

Is Python better to start with if you are totally new to programming ? 

Id say any language really is but Python is pretty easy and it comes with loads of good utilities in popular coding tools like Visual Studio Code to get started. 

[Bluebird Hewitt]

I was having a student meet up today and when I asked about any practical elements of my degree course, one of them happens to be Python. 

My colleague has given me a link to learn Python a little bit online, but wondering if anyone has any advise to someone completely new to this. 

[nudge]

5 minutes ago, Bluebird Hewitt said:

I was having a student meet up today and when I asked about any practical elements of my degree course, one of them happens to be Python. 

My colleague has given me a link to learn Python a little bit online, but wondering if anyone has any advise to someone completely new to this. 

What exactly are you looking for? Python is a multi-purpose language, so I'd say it depends a lot on what you want to focus on. 

[Bluebird Hewitt]

3 minutes ago, nudge said:

What exactly are you looking for? Python is a multi-purpose language, so I'd say it depends a lot on what you want to focus on. 

No idea at the moment. I haven't started the course just yet (that's from the 27th September). 

To give an idea of what I'm doing (copied and pasted from an e-mail I received). 

Semester 1 modules are Networks and Cybersecurity Essentials and Learning in the Digital Era.

Semester 2 modules are Introduction to Web and Databases and Software Development

I should know more during my induction, which is on Monday. 

[nudge]

1 minute ago, Bluebird Hewitt said:

No idea at the moment. I haven't started the course just yet (that's from the 27th September). 

To give an idea of what I'm doing (copied and pasted from an e-mail I received). 

Semester 1 modules are Networks and Cybersecurity Essentials and Learning in the Digital Era.

Semester 2 modules are Introduction to Web and Databases and Software Development

I should know more during my induction, which is on Monday. 

In that case, I would just start with one of the online interactive courses on codecademy or datacamp in order to learn the basic syntax and logic. 

I personally started with the online MITx course as I wanted more structure in my learning, but in the end, I found I learned by far more from starting small projects and learning as I went (using google, stackexchange, github and official documentation as resources to learn new things).

[Bluebird Hewitt]

3 minutes ago, nudge said:

In that case, I would just start with one of the online interactive courses on codecademy or datacamp in order to learn the basic syntax and logic. 

I personally started with the online MITx course as I wanted more structure in my learning, but in the end, I found I learned by far more from starting small projects and learning as I went (using google, stackexchange, github and official documentation as resources to learn new things).

Cheers for this. Funnily enough, the one in bold is what my colleague provided, but I'll note the other ones as well. 

[nudge]

Just now, Bluebird Hewitt said:

Cheers for this. Funnily enough, the one in bold is what my colleague provided, but I'll note the other ones as well. 

Yeah it's a good choice, I think there's not much difference in which platform you choose to start with... Good luck! I really enjoy it myself.

[Mel81x+]

7 hours ago, Bluebird Hewitt said:

No idea at the moment. I haven't started the course just yet (that's from the 27th September). 

To give an idea of what I'm doing (copied and pasted from an e-mail I received). 

Semester 1 modules are Networks and Cybersecurity Essentials and Learning in the Digital Era.

Semester 2 modules are Introduction to Web and Databases and Software Development

I should know more during my induction, which is on Monday. 

When I see semester courses setup that way it leans towards coding fundamentals in Sem 1 (you probably wont write as much code if at all) and then Sem 2 will be learning to use things like "flask", etc. I'd actually spend time on other sites to get a bit of understanding Python before going into Sem2. Python is super versatile and actually easy to understand once you get your head around the syntax. Have you programmed in any other language before?

[Bluebird Hewitt]

2 hours ago, Mel81x said:

When I see semester courses setup that way it leans towards coding fundamentals in Sem 1 (you probably wont write as much code if at all) and then Sem 2 will be learning to use things like "flask", etc. I'd actually spend time on other sites to get a bit of understanding Python before going into Sem2. Python is super versatile and actually easy to understand once you get your head around the syntax. Have you programmed in any other language before?

None whatsoever. Programming is completely new to me. 

From what I saw for the course as a whole, the first year is generic regardless of which area you study (mine is Data and Information Systems), but you study more in your area from the second year onwards. 

[Mel81x+]

15 minutes ago, Bluebird Hewitt said:

None whatsoever. Programming is completely new to me. 

From what I saw for the course as a whole, the first year is generic regardless of which area you study (mine is Data and Information Systems), but you study more in your area from the second year onwards. 

That makes sense considering its something that you'll will be getting into for data systems. I don't think you'll have a problem at all and if you have homework issues @nudge can help

[Bluebird Hewitt]

39 minutes ago, Mel81x said:

That makes sense considering its something that you'll will be getting into for data systems. I don't think you'll have a problem at all and if you have homework issues @nudge can help

Cheers Mel and @nudge. Much appreciated.