umask for new created files with NIS users
Added by Mario Dejung over 13 years ago
I just wonder, how the files get created by appion.
I use a NIS system for my user accounts. When I create a stack for example, the files get created by the user but with other permissions then a local system user. So the problem is, I can not read the queue logfile or any image, since the permissions are 600 and the apache user can not display them on the website.
The default umask for the user is 0022 independent of active or inactive user shell. How can it be, that the permissions of NIS users are different then for example a local system user. I could change the umask for the apache user but this will not fix the problem, since the owner is a different user.
Hope somebody has tried this with a NIS user accounts, in the meantime I will create local system users and switch of the NIS.
Replies (11)
RE: umask for new created files with NIS users - Added by Christopher Irving over 13 years ago
Mario,
The only umask Appion ever tries to set is 002, never 066 and where a users login info comes from makes no difference to Appion. All of our users here at Scripps use NIS accounts and NFS mounted file systems and we don't have this problem.
If all your files are getting set to 600 and your sure the umask of the user running the Appion scripts is set to 022 then this is probably a files system issue. Where are you saving your files? Is it a local file system, an NFS mount, an CIFS mount? Are your NIS users saving to an NFS mounted home directory and locally created users saving somewhere to a local files system?
Give us some more information and we can probably help you out but I'm almost certain this isn't an Appion issue.
-Christopher
RE: umask for new created files with NIS users - Added by Mario Dejung over 13 years ago
Hi Christopher,
thanks for your fast reply.
I created a LVM Volume made of two hard drives with ext3 filesystem and mounted it to /myamiImages, the fstab options look like:
/dev/emdata/emdata /myamiImages ext3 defaults 0 0
The home directories of the users mounted via nfs but this should not make any problems
zoo2cluster26:/srv/data/home /data/home nfs async,rw 0 0
When I use dogpicker for example, the folder will look like this. I also used the command umask to show my umask:
0 dejungma@appion /myamiImages/appion/sample/extract/dogrun6 $ ls l 1 dejungma users 144 25. Jan 08:18 dog_image_timing.dat
insgesamt 132
-rw------rw------ 1 dejungma users 74962 25. Jan 08:18 dogmap.jpgrw------ 1 dejungma users 497 25. Jan 08:18 dogPicker.donedictrw------ 1 dejungma users 446 25. Jan 08:18 dogPicker.logrw-r--r- 1 dejungma users 555 25. Jan 08:17 dogrun6.appionsub.jobrw------ 1 dejungma users 232 25. Jan 08:17 dogrun6.appionsub.job.o27rw------ 1 dejungma users 18042 25. Jan 08:18 dogrun6.appionsub.log
drwx------ 2 dejungma users 4096 25. Jan 08:18 jpgs
drwx------ 2 dejungma users 4096 25. Jan 08:18 maps
drwx------ 2 dejungma users 4096 25. Jan 08:18 pikfiles
0 dejungma@appion /myamiImages/appion/sample/extract/dogrun6 $ umask
0022
How is apache executing the files as a specific user? Is there a special lib to do this.
RE: umask for new created files with NIS users - Added by Christopher Irving over 13 years ago
How is apache executing the files as a specific user? Is there a special lib to do this.
The php script that's running on your Apache server is logging in to you processing host via ssh using the username and password you supplied at the top of the processing page (in your case, the dogpicker page).
So you're running everything on one machine, is that correct?
What shell are you using (bash, etc)?
-Christopher
RE: umask for new created files with NIS users - Added by Mario Dejung over 13 years ago
Hi Christopher,
yes, I use everything on one machine. I used the autoinstallscript to install appion after a fresh installation. The only changes I mad was switching to user_login=true, so the users have to login to the webinterface (but I also get the same results if I switch it to false again) and after the installation, I created a LVM of two harddrives with ext3 file system and mounted them to the /myamiImages folder.
I have to check, if it works with the same harddrive, so without mounting the logical volume...
I have have to check if maybe php is the problem in my case, but I do not think so. Is this a system call or is it libssh2 library? I wonder, the jobfile gets created with 644, only the other files have 600 as permission.
I use a bash shell by the way. And also for active or interactive shell the umask is 0022.
kind regards
Mario
RE: umask for new created files with NIS users - Added by Christopher Irving over 13 years ago
I have to check, if it works with the same harddrive, so without mounting the logical volume...
The logical volume has nothing to do with file permissions, it's at a lower level than the filesystem.
Is this a system call or is it libssh2 library?
It uses the libssh2 library to login both when creating the log file and when running the appion scripts.
I wonder, the jobfile gets created with 644, only the other files have 600 as permission
Yes, I noticed that and thats why I think it has to do with your user environment. Could you send me a copy of all your .bash* files for the user dejungma.
-Christopher
RE: umask for new created files with NIS users - Added by Mario Dejung over 13 years ago
Hi Christopher,
these are all my bash files for user dejungma...
RE: umask for new created files with NIS users - Added by Christopher Irving over 13 years ago
Mario,
Can you try moving your .bash_profile and .bashrc file aside and then running dogpicker again as dejungma.
So do something like:
cp .bash_profile tmp.bash_profile
cp .bashrc tmp.bashrc
Then try running dogpicker and send me the listing of the directory created.
Thanks,
Christopher
RE: umask for new created files with NIS users - Added by Mario Dejung over 13 years ago
Hi Christopher
I've made the changes and run dogpicker, the results are still the same:
ls al /myamiImages/appion/sample/extract/dogrun16/ 1 dejungma users 144 28. Jan 08:13 dog_image_timing.dat
insgesamt 136
drwxr-xr-x 5 dejungma users 4096 28. Jan 08:14 .
drwxrwxrwx 18 root root 4096 28. Jan 08:13 ..
-rw------rw------ 1 dejungma users 74962 28. Jan 08:13 dogmap.jpgrw------ 1 dejungma users 497 28. Jan 08:14 dogPicker.donedictrw------ 1 dejungma users 448 28. Jan 08:14 dogPicker.logrw-r--r- 1 dejungma users 559 28. Jan 08:13 dogrun16.appionsub.jobrw------ 1 dejungma users 0 28. Jan 08:13 dogrun16.appionsub.job.o42rw------ 1 dejungma users 18061 28. Jan 08:14 dogrun16.appionsub.log
drwx------ 2 dejungma users 4096 28. Jan 08:14 jpgs
drwx------ 2 dejungma users 4096 28. Jan 08:14 maps
drwx------ 2 dejungma users 4096 28. Jan 08:14 pikfiles
Here are all the bash files remaining in my home directory:
find ~/ -name '*bash*'
/data/home/dejungma/.bash
/data/home/dejungma/.bash/bash-vars
/data/home/dejungma/.bash/bashrc
/data/home/dejungma/.bash_history
/data/home/dejungma/tmp.bash_profile
/data/home/dejungma/temp.bashrc
I made a change in /var/www/html/myamiweb/processing/inc/processing.inc line 947 and 941. I added the umask command to apcmd variable so I got the output of umask in a pbslogfile. It says 0077:
cat dogrun18.appionsub.job.o44
0077
The .basrc and .bash_profile are still renamed. I added the umask 0022 command to these line (line 947: $apcmd = "umask 0022;webcaller.py '".$filtcommand."' $rundir/$logfile"; and line 941: $apcmd[] = "umask 0022; webcaller.py '".$filtcommand."' $rundir/$logfile"."\n";) and this solved the problem for now.
I'm still not sure where exactly the problem is and don't know it it's the best way to do it, but maybe you have a better solution...
Kind regards
RE: umask for new created files with NIS users - Added by Christopher Irving over 13 years ago
Have you tried using a very basic .bash_profile file. It should only have the line
umask 022
The only way I've been able to recreate your problem is by setting the umask to 077 in one of the shell dot files (.bash_profile, .bashrc, etc.). I've also searched the appion source code multiple times and I'm certain that it is not getting set by appion. Your own solution which sets the umask before any of the appion scripts are called confirms this point. If appion were setting the umask this solution would not have worked.
If using the basic .bash_profile doesn't work that means the umask is getting reset after the shell files are sourced and before appion runs. That basically only leaves ssh or torque. Neither of these are very likely especially if you're using a freshly installed Centos machine.
-Christopher
RE: umask for new created files with NIS users - Added by Mario Dejung over 13 years ago
Hi Christopher,
the tip with the basic .bash_profile worked. I have to write our system support again. The first time they told me that the problem should belong to appion, since they only set the umask of 022. But now I can prove that the basic .bash_profile works fine and everything gets messed-up when I use the default system .bash_profile...
I will write the solution here after I get new answers from my support.
Kind Regards
Mario
RE: umask for new created files with NIS users - Added by Christopher Irving over 13 years ago
Mario,
That suggests that the problem must be with the /etc/profile file or a file it sources.
-Christopher