Using syntax or python to run script on all files

Moderators: statman, Analyst Techy, andris, Fierce, GerineL, Smash

w47191
Posts: 1
Joined: Thu Jan 21, 2010 3:53 pm

Using syntax or python to run script on all files

Postby w47191 » Sun Nov 18, 2012 2:58 am

Hi,

I am trying to figure out a way to tell spss syntax to run itself on all files in a certain directory and then either create one large file where the outputs are merged or to create separate output files. A friend sent me this script which seems to work in python but I am having trouble using it in SPSS:

import SpssClient
SpssClient.StartClient()

for files in os.listdir("."):
f = open(files)
outPutOfF = f.read()

print outPutOfF
print ""

work = open("C:/Users/kang/processed/" + files + "_workfile.txt", "a")
work.write( "Content of outPutOfF is: " + outPutOfF )
f.close()
work.close()
SpssClient.StopClient()

The script that I made with spss syntax is designed to open individual files, format them correctly, derive a mean, and then save an output of each individual file. The problem is that I have to change the name of the file that it is accessing each time I run it, so that it will eventually run all of the files I need it to. However, I would like this process automated. The spss syntax script I use is:


GET DATA
/TYPE=TXT
/FILE='F:\Form & Motion Discrimination\RAW\10005_dir.txt'
/DELCASE=LINE
/DELIMITERS=","
/ARRANGEMENT=DELIMITED
/FIRSTCASE=1
/IMPORTCASE=ALL
/VARIABLES=
Trial F27.1
Vis F16.2
V3 F2.0
V4 F2.0
V5 F5.0
V6 F2.0.
CACHE.
EXECUTE.
DATASET NAME DataSet1 WINDOW=FRONT.

SELECT IF Trial = 45 OR Trial = 46 OR Trial = 47 OR Trial = 48 OR Trial = 49 OR Trial = 50.
EXECUTE.

SAVE OUTFILE = 'F:\Form & Motion Discrimination\Data\RAW\10005_dir.sav'
/keep Vis.
EXECUTE.

GET FILE='F:\Form & Motion Discrimination\Data\RAW\10005_dir.sav'.
EXECUTE.

AGGREGATE
/OUTFILE='F:\Form & Motion Discrimination\Data\TEMP\RT.sav'
/RT = MEAN(Vis).

Any suggestions would be appreciated. I have some experience using the spss syntax but very little experience with python.

Oh and I am using spss 20.
pythonforspss.org
Posts: 116
Joined: Sat Oct 06, 2012 6:21 am

Re: Using syntax or python to run script on all files

Postby pythonforspss.org » Thu Dec 06, 2012 5:30 am

Dear w47191,

You are definitely on the right track! The basic idea is this:

*1. Ask Python to look in a folder for all files with some extension and collect the paths for you. This avoids having to type in filenames manually.

begin program.
rdir=r"d:\temp" # Please specify folder holding .txt files
fils=[fil for fil in os.listdir(rdir) if fil.endswith(".txt")]
print fils
end program.

*2. Loop through this list of files.

begin program.
for fil in fils:
spss.Submit("""
[SPSS Commands go here]
""")
end program.

An even better approach may be to

*1. Fetch and merge the contents of all .txt files into Python
*2. Submit those to SPSS as a new DataSet
*3. Then do your data edits.

Hope that helps!
Kind regards,

Ruben Geert van den Berg
http://www.spss-tutorials.com

Who is online

Users browsing this forum: No registered users and 2 guests

cron