# Customize passenger environment for apps

**URL:** <https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465>\
**Category:** Get Help\
**Tags:** ondemand2, question\
**Created:** [February 7, 2023, 12:43am UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465 "2023-02-07T00:43:27Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![milberg](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@milberg](https://discourse.openondemand.org/u/milberg)\
**Post date:** [February 7, 2023, 12:43am UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/1 "2023-02-07T00:43:27Z")

</div>

Background: Setting paths for tmp/scratch files for apps by setting $TMPDIR in the app is working (for tmp files of app on compute node).  
New (different) issue: `/tmp` on OOD webservers is filling up, and seems to be from passenger tmp files. (`/tmp/passenger.NAME` don’t seem very large, but `/tmp/RackMultipartNAME.dta` have filled up `/tmp` in some cases)  
Question: Is there a way to set `passenger_data_buffer_dir` for apps?  
I see that a number of values for the passenger environment are set in `/var/lib/ondemand-nginx/config/puns/USER.conf`, some of which come from `/etc/ood/config/nginx_stage.yml`, but `passenger_data_buffer_dir` isn’t among them.  
Would this idea resolve the issue? Is there a better approach?

---

<div class="post-metadata">

**Author:** ![travert](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/travert/32/3074_2.png) [@travert](https://discourse.openondemand.org/u/travert)\
**Post date:** [February 7, 2023, 3:02pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/2 "2023-02-07T15:02:25Z")

</div>

Hi and thanks for the question.

The simplest way to set something like this is to look at the `nginx_stage` docs, and notice the `passenger_options` configuration option:  
[https://osc.github.io/ood-documentation/latest/reference/files/nginx-stage-yml.html?highlight=passenger\_options](https://osc.github.io/ood-documentation/latest/reference/files/nginx-stage-yml.html?highlight=passenger_options)

You can add configurations from [Configuration reference - Nginx - Passenger Library](https://www.phusionpassenger.com/library/config/nginx/reference/#passenger_data_buffer_dir) by simply throwing them into this hash and they will be picked up and applied.

Let me know if this doesn’t work as expected though of if you need more clarification.

---

<div class="post-metadata">

**Author:** ![milberg](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@milberg](https://discourse.openondemand.org/u/milberg)\
**Post date:** [February 7, 2023, 8:31pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/3 "2023-02-07T20:31:14Z")

</div>

Thanks for the information!

Setting `passenger_data_buffer_dir` in `passenger_options` in `nginx_stage.yml` did result in the value being set in `/var/lib/ondemand-nginx/config/puns/USER.conf`. But passenger tmp files are still being written in `/tmp` rather than the path set. The path isn’t available at boot time as it is NFS-mounted. After restarting the httpd service on the server, and ‘Restart Web Server’ in the website, the new path still isn’t used. (I have confirmed that the path exists, is writable by all (permission 777), and SELinux is disabled.)

Is setting `passenger_data_buffer_dir` an incorrect approach to the issue, or might something be preventing it from working?

---

<div class="post-metadata">

**Author:** ![travert](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/travert/32/3074_2.png) [@travert](https://discourse.openondemand.org/u/travert)\
**Post date:** [February 8, 2023, 5:42pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/4 "2023-02-08T17:42:38Z")

</div>

It’s possible it’s due to the NSF mounting. Is there a way you can define the `tmp` but use something on the local fs rather than the NSF mounted fs? That would at least show you if it is to do with the boot order.

The restarting of the PUN should have been sufficient to get this to work, so knowing about whether that fs not being there at boot is the key here.

---

<div class="post-metadata">

**Author:** ![milberg](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@milberg](https://discourse.openondemand.org/u/milberg)\
**Post date:** [February 9, 2023, 9:09pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/5 "2023-02-09T21:09:19Z")

</div>

Thanks for the suggestion. I tried using a local path and the passenger tmp files were still written in `/tmp` rather than the value of `passenger_data_buffer_dir`.

---

<div class="post-metadata">

**Author:** ![travert](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/travert/32/3074_2.png) [@travert](https://discourse.openondemand.org/u/travert)\
**Post date:** [February 9, 2023, 9:19pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/6 "2023-02-09T21:19:39Z")

</div>

Ok, sorry for the issue.

What version of OOD are you running and what OS?

I’ll have to look at this on my end to try and see what is happening as that should work.

---

<div class="post-metadata">

**Author:** ![milberg](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@milberg](https://discourse.openondemand.org/u/milberg)\
**Post date:** [February 9, 2023, 9:21pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/7 "2023-02-09T21:21:58Z")

</div>

It is OOD v2.0.20 on RHEL8.

---

<div class="post-metadata">

**Author:** ![milberg](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@milberg](https://discourse.openondemand.org/u/milberg)\
**Post date:** [February 9, 2023, 9:38pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/8 "2023-02-09T21:38:13Z")

</div>

Also tested with OOD 2.0.28 on RHEL8 (with same result).

---

<div class="post-metadata">

**Author:** ![travert](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/travert/32/3074_2.png) [@travert](https://discourse.openondemand.org/u/travert)\
**Post date:** [February 10, 2023, 3:29pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/9 "2023-02-10T15:29:58Z")

</div>

Ok, so it looks like Passenger does not quite play as nice as we’d hoped. They use a variable within the application called `Tempfile` that it does look like we can work with, but only using the env var for the dashboard of `TMPDIR`.

The solution could be instead of telling passenger to handle this, instead set it for the dashboard itself. This can be done by having a `/etc/ood/config/apps/dashboard/env` file that then has an entry for:

```auto
TMPDIR=/your/tmp

```

This _should_ work with the NFS mounted filesystem and have these files land where you are hoping.

Please let me know if this works as expected for you.

---

<div class="post-metadata">

**Author:** ![milberg](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@milberg](https://discourse.openondemand.org/u/milberg)\
**Post date:** [February 10, 2023, 10:20pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/10 "2023-02-10T22:20:29Z")

</div>

Thanks for investigating. I tried adding a line for `TMPDIR` to `/etc/ood/config/apps/dashboard/env` in a few different ways, and for all attempts it kept using /tmp rather than the path I set. I tried both `TMPDIR=/your/tmp` and `TMPDIR="/your/tmp"`, for both local and NFS mounted filesystems.

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [February 13, 2023, 2:41pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/11 "2023-02-13T14:41:05Z")

</div>

I confirmed this worked in my dev environment. I’d check 2 things

- bounce your web server when you make a config change for it to pickup any new configuration.
- check the permissions on the directory you set `TMPDIR` to.

Here’s how `Ruby` tries to find a tmpdir and you can see it requires it to be a directory that’s not only writable but world writable (with no sticky bit set).

> <https://github.com/ruby/ruby/blob/bc6e587e86a4a4a46af3b1c2fbe1ff3b399735a6/lib/tmpdir.rb#L2296+>

Here’s what I saw in my logs to confirm. You can see the `@tempfile` paramter is in my `$HOME`

```auto
  Parameters: {"relativePath"=>"null", "name"=>"core.21946", 
"type"=>"application/octet-stream", "parent"=>"/users/PZS0714/johrstrom", 
"file"=>#<ActionDispatch::Http::UploadedFile:0x00007f2df4d252e8 
@tempfile=#<Tempfile:/users/PZS0714/johrstrom/temp/tmp_test/RackMultipart20230210-51306-v9vxte.21946>, 
@original_filename="core.21946", @content_type="application/octet-stream", 
@headers="Content-Disposition: form-data; name=\"file\"; 
filename=\"core.21946\"\r\nContent-Type: application/octet-stream\r\n">, 
"fs"=>"fs"}

```

---

<div class="post-metadata">

**Author:** ![milberg](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@milberg](https://discourse.openondemand.org/u/milberg)\
**Post date:** [February 15, 2023, 5:30pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/12 "2023-02-15T17:30:54Z")

</div>

Thanks for the information. I have done some further testing based on it.

- For all tests, I restarted the httpd service and did ‘Restart Web Server’ in the browser. A few times, I also rebooted the server.
- The Ruby fragment shows that the tmpdir must be writable, and if it’s world-writable then the sticky bit must be set. For one test case, the sticky bit wasn’t set when it was needed, so I corrected that.

I copied the tmpdir method into a test script which I ran on the server with different values of TMPDIR set. It seems to give the expected result for all cases. Script test.rb is:

```auto
#!/usr/bin/ruby

class D

  @@systmpdir ||= defined?(Etc.systmpdir) ? Etc.systmpdir : '/tmp'

  def gettmp()
    ['TMPDIR', 'TMP', 'TEMP', ['system temporary path', @@systmpdir], ['/tmp']*2, ['.']*2].find do |name, dir|
      unless dir
        next if !(dir = ENV[name]) or dir.empty?
      end
      dir = File.expand_path(dir)
      stat = File.stat(dir) rescue next
      case
        when !stat.directory?
          warn "#{name} is not a directory: #{dir}"
        when !stat.writable?
          warn "#{name} is not writable: #{dir}"
        when stat.world_writable? && !stat.sticky?
          warn "#{name} is world-writable: #{dir}"
        else
          break dir
      end
    end or raise ArgumentError, "could not find a temporary directory"
  end
end

d = D.new
mydir = d.gettmp()
puts "tmpdir is: #{mydir}";   

```

I get these results:

```auto
$ echo $TMPDIR

$ ruby test.rb 
tmpdir is: /tmp
$ export TMPDIR=/tmp/test
$ ruby test.rb 
tmpdir is: /tmp/test
$ export TMPDIR=/exists/tmp 
$ ruby test.rb 
tmpdir is: /exists/tmp
$ export TMPDIR=/doesnt/exist         
$ ruby test.rb 
tmpdir is: /tmp

```

Logs like those you show look like they would be helpful. Where can I find them? (I only know where the Apache and Nginx logs are.)

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [February 15, 2023, 6:27pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/13 "2023-02-15T18:27:43Z")

</div>

They’re from development logs. I’m not sure how to enable them in production, as they’d be very verbose as they log a lot of information.

In any case, you could add an initializer at this location that could just print what we’re looking for.

```auto
# /etc/ood/config/apps/dashboard/initializers/tmp_debug.rb

puts "TMPDIR is currently #{Dir.tmpdir}"

```

And you’ll see output in the Nginx logs like:

```auto
App 127355 output: TMPDIR is currently /tmp

# ... or if it works
App 129486 output: TMPDIR is currently /users/PZS0714/johrstrom/temp/tmp_test

```

There’s also 1 more thing that may be getting in our way. The environment variables in `/etc/ood/config/apps/dashboard/env` won’t override _existing_ environment variables. That is, if your system has somehow set `TMPDIR`, it won’t be overwritten by just the `env` file.

To overcome this, you can set TMPDIR in `/etc/ood/config/apps/dashboard/env.overload`. `overload` files aren’t documented because they’re mostly a development feature. For example you could override `$HOME` or `$USER` to some disastrous effect, so use this with caution.

---

<div class="post-metadata">

**Author:** ![milberg](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@milberg](https://discourse.openondemand.org/u/milberg)\
**Post date:** [February 15, 2023, 9:11pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/14 "2023-02-15T21:11:46Z")

</div>

I tried that and get:

```auto
App 34551 output: TMPDIR is currently /my/expected/path

```

So that part seems to be working. But directories `passenger.NAME` are still being created in `/tmp`. (I’ve been checking these since they’re always created, although `RackMultipartNAME.dta` are more important to place in a different location.) Should directories `passenger.NAME` be created in TMPDIR, or am I checking the wrong thing?

---

<div class="post-metadata">

**Author:** ![jeff.ohrstrom](https://sea1.discourse-cdn.com/flex015/user_avatar/discourse.openondemand.org/jeff.ohrstrom/32/136_2.png) [@jeff.ohrstrom](https://discourse.openondemand.org/u/jeff.ohrstrom)\
**Post date:** [February 15, 2023, 9:44pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/15 "2023-02-15T21:44:35Z")

</div>

> [@milberg](#):
>
> So that part seems to be working. But directories `passenger.NAME` are still being created in `/tmp`. (I’ve been checking these since they’re always created, although `RackMultipartNAME.dta` are more important to place in a different location.) Should directories `passenger.NAME` be created in TMPDIR, or am I checking the wrong thing?

Sorry - I should have been clearer. Yes what you are seeing is what I anticipated. Setting the TMPDIR in that location is only for the dashboard, which is lower in the stack than Passenger. In any case, I thought the `RackMultipartNAME` are the real culprit for blowing up any drive, and so was focusing on that.

For the other settings/other temporary passenger things, maybe you’re looking for this or at least I’d search that page for `tmp` to indicate how you can migrate _everything_ off of that directory.

[https://www.phusionpassenger.com/library/config/nginx/reference/#passenger\_instance\_registry\_dir](https://www.phusionpassenger.com/library/config/nginx/reference/#passenger_instance_registry_dir)

---

<div class="post-metadata">

**Author:** ![milberg](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@milberg](https://discourse.openondemand.org/u/milberg)\
**Post date:** [February 15, 2023, 9:45pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/16 "2023-02-15T21:45:40Z")

</div>

I also tried putting the `TMPDIR` line from `/etc/ood/config/apps/dashboard/env` in `/etc/ood/config/apps/dashboard/env.overload`. This doesn’t change the behavior. Still get correct value for `TMPDIR` in the logs.

---

<div class="post-metadata">

**Author:** ![milberg](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@milberg](https://discourse.openondemand.org/u/milberg)\
**Post date:** [February 15, 2023, 9:47pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/17 "2023-02-15T21:47:30Z")

</div>

Thanks for clarifying. Since it seems like config should work for `RackMultipartNAME`, I’ll get someone to test that.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex015/uploads/osc/original/2X/b/bae70bd0ed39a3ae769c2108155f4cb3e9da8385.png) [@system](https://discourse.openondemand.org/u/system)\
**Post date:** [August 14, 2023, 9:48pm UTC](https://discourse.openondemand.org/t/customize-passenger-environment-for-apps/2465/18 "2023-08-14T21:48:10Z")

</div>

This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.