Commit Graph

130 Commits

Author SHA1 Message Date
David Taylor
b6b27bc383
DEV: Improve auto-restart parent process detection logic (#13068)
The auto restart logic was sending a USR2 to the parent process without checking what the parent process actually was. In some situations, it might not be the `bin/unicorn` supervisor.

This commit switches to use a global variable for the supervisor PID. This will be much less prone to unexpected behavior.
2021-05-14 18:17:31 +01:00
Jeff Wong
7a0e5c8cb9
FIX: properly filter ember-cli args (#13011)
Was previously rejecting all args.
2021-05-10 10:39:54 -07:00
Penar Musaraj
49de5dadf7
DEV: Ember CLI postinstall hook and shortcut (#12987) 2021-05-10 09:34:24 -04:00
Jarek Radosz
53c06442ef
DEV: Clean up .gitignore (#12981)
(inspired by martin-brennan's recent adventure with `tags` exclusion)

**tl;dr: some of it is obsolete, some is dev-env specific and should live in *your* ~/.gitignore.**

---

To enable global gitignore (`~/.gitignore`) run:
```bash
git config --global core.excludesFile '~/.gitignore'
```

Then create that file and add your VIM/VSCode/Sublime/Emacs/Eclipse/RubyMine/JetBrains/macOS/Arch/Windows 95-specific stuff.

---

Reasons for removal:

* `bin` - never really ignored, all the files are checked in
* `.DS_Store`, `._.DS_Store` - OS specific
* `.sass-cache/*`, `/.bundle`, `/bundle/*`, `/cache`, `/logfile`, `!/plugins/discourse-nginx-performance-report`, `MiniProfiler/Ruby/rack-mini-profiler-2.0.1a.gem`, `config/fog_credentials.yml`, `/public/stylesheet-cache/*`, `script/download_db`, `script/refresh_db`, `config/version.rb` - no longer used?
* `/log/*.log` - covered by /log
* `bootsnap-load-path-cache`, `bootsnap-compile-cache` - bootsnap now keeps its ~~trash~~ temporary files in `/tmp`
* `/.project`, `/.buildpath`, `/.byebug_history`, `/.idea`, `discourse.sublime-workspace`, `*~`, `*.swp`, `*.swo`, `*.swm`, `config/multisite1.yml`, `.rvmrc`, `.ruby-version`, `.ruby-gemset`, `.rbenv`, `bundler_stubs/*`, `*.db`, `*.iml`, `*.swn`, `/package-lock.json`, `.vscode`, `.envrc`, `tags` - dev-env specific, see the note at the top of the commit
2021-05-10 13:43:13 +02:00
Jeff Wong
bc3c3d56e0
DEV: do not cd into yarn dir, use --cwd argument to yarn instead (#12986)
* DEV: do not cd into yarn dir, use --cwd argument to yarn instead

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2021-05-07 09:58:57 -07:00
David Taylor
ac1d344368
DEV: Update server startup warning to be more specific (#12949)
Using `bundle exec` will slow down server startup by at least 0.5s. `bin/unicorn` has built-in handling of bundler dependencies, so it is better to launch `bin/rails s` or `bin/unicorn` directly.
2021-05-05 13:33:11 +01:00
David Taylor
0188d53f3a
DEV: Add performance warning message when running rails s (#12912)
In Discourse, `rails s` ultimately launches the `bin/unicorn` script. However, the overhead of `rails` launching `bin/rails`, and then in turn `bin/unicorn` can be non-trivial. Therefore it is much better to run `bin/unicorn` directly.

This commit prints a warning message to STDERR when `rails s` is used. Functionality is unchanged.
2021-04-30 11:50:06 +01:00
Robin Ward
51f872f13a
DEV: Require Ember CLI to be used in development mode (#12738)
We really want to encourage all developers to use Ember CLI for local
development and testing. This will display an error page if they are not
with instructions on how to start the local server.

To disable it, you can set `NO_EMBER_CLI=1` as an ENV variable
2021-04-29 14:13:36 -04:00
Sam
307cb1d114
DEV: add helper script to run ember cli (#12005)
This little helper script allows for easy ember cli development.

To see the options run `bin/ember-cli -h`

It allows you to proxy try.discourse.org with the `bin/ember-cli --try`
switch, which effectively allows for some development without a rails installed.

It passes on arguments to ember-cli so you can customize port and so on.

It makes the assumption that on local people are using `bin/unicorn` for
development. (it includes some extra discourse specific helpers)
2021-02-09 16:33:14 +11:00
Arpit Jalan
bbf6cc4f18
DEV: require fileutils gem on boot (#11542) 2020-12-21 16:31:06 +11:00
Robin Ward
2eb9c0f3dd FIX: notify_file_change was outputting a command to vim
The check for `socat` needs to be silenced too, or vim will output the
path to `socat` every time you save a file.
2020-12-10 10:03:57 -05:00
Alessandro Bertacco
dc2f5064e2
DEV: support for relative symlinks for plugins in docker dev (#10955)
These changes gives the possibility to use relative symlinks for plugins. Previously only absolute symlinks was supported.
2020-10-21 09:39:53 +11:00
David Taylor
46d1c91e1a
DEV: Make notify_file_change fallback to nc when socat missing (#10903) 2020-10-13 17:03:24 +01:00
Kane York
4635be10c8 DEV: Add docker cleanup script to d/ folder 2020-03-01 12:09:07 -08:00
Sam Saffron
d8412f409a DEV: resolve symlinks in docker dev
symlinks where not fully resolved leading to docker not booting when you
had symlinks in the plugins directory pointing at relative paths.
2020-01-13 10:33:34 +11:00
Martin Brennan
aa2b3e71b4 Minor refactor of bin/unicorn cache clean
* move git ls-files for plugins folder into one line
2019-11-20 08:47:51 +10:00
Sam Saffron
9652fdd111 DEV: if tmp is missing create it
Some scripts and people may be nuking the entire tmp folder
2019-11-20 09:29:34 +11:00
Martin Brennan
93d7abe372
FIX: Automatically recover from bad sprockets cache in development (#8364)
We were having issues in development mode where the JS code had errors due to a bad cache. When starting a server in development mode in bin/unicorn we now get the git sha of the discourse HEAD and get a git sha of all plugins, and store them in a file. If the sha has changed then we delete tmp/cache to refresh the assets cache.
2019-11-19 09:15:09 +10:00
Sam Saffron
ff33899323 FEATURE: allow publishing docker dev either locally or net wide
Previously we had no control over how internal ports in the containter got
published.

Following UNICORN_BIND_ALL=true setting this broke docker dev env and exposed
this weakness.

The new `d/boot_dev` will only export on localhost, if you wish to export
network with use `d/boot_dev -p`
2019-11-04 12:51:35 +11:00
Mark VanLandingham
9b4aba0d39
DEV: support --fail-fast in bin/turbo_rspec (#8170)
* [WIP] - default turbo spec env to test

* FEATURE: support for --fast-fail in bin/turbo_rspec

* fast-fail -> fail_fast to match rspec

* Moved thread killing outside of fail-fast check

* Removed failure_count incrementation from fast_fail_met
2019-10-09 09:40:06 -05:00
Sam Saffron
7c084c7cab DEV: allow USR2 to restart unicorn in dev
Well all this does is amends a commit comment cause I was over eager and
pushed previous commit.

This is the comment I wanted....

This allows `pkill -USR2 -f 'ruby bin/unicorn'` to restart current unicorn

It is handy if you want to bind a keyboard shortcut for unicorn restarts
in dev.

By doing so you can avoid finding the terminal, hitting ctrl-c and then
hitting up, enter, heading back to browser.

It saves time.

Automate stuff, you will not regret it...
2019-09-11 17:45:47 +10:00
Sam Saffron
fb8d1f35a4 DEV: support USR2 to restart unicorn in dev mode
In dev mode sending USR2 to the unicorn master supervisor process will
restart unicorn.

This allows a simple script like this to restart unicorn in dev:

```
#!/usr/bin/env bash

kill -s USR2 `ps aux | grep ruby\ bin\/unicorn | grep -v grep | awk '{print $2}'`
```
2019-09-11 17:35:58 +10:00
hawm
16681cb648 Fix docker image name (#8058) 2019-09-04 16:19:48 +10:00
Kyle Zhao
2fbafd077c DEV: Mount plugin symlinks to dev docker container (#8002)
When developing using docker, in order to support symlinks in the
`plugins/` directory, this reads the symlinks' values and mounts them to
the dev docker container.
2019-08-14 18:13:01 +10:00
Kyle Zhao
e2df331fdb DEV: default to test env when running d/rspec (#7962) 2019-08-02 16:40:46 +10:00
Daniel Waterworth
c3db5925a8 FIX: Turbo tests exit codes 2019-07-09 08:51:23 +01:00
Daniel Waterworth
d6aa92e98e DEV: Add a verbose option to ./bin/turbo_rspec 2019-06-27 15:49:21 +01:00
Sam Saffron
fc84e23b71 DEV: allow bin/turbo_tests to run tests without params 2019-06-21 11:33:22 +10:00
Daniel Waterworth
e18ce56f4b DEV: Add a new way to run specs in parallel with better output (#7778)
* DEV: Add a new way to run specs in parallel with better output

This commit:

 1. adds a new executable, `bin/interleaved_rspec` which works much like
    `rspec`, but runs the tests in parallel.

 2. adds a rake task, `rake interleaved:spec` which runs the whole test
    suite.

 3. makes autospec use this new wrapper by default. You can disable this
    by running `PARALLEL_SPEC=0 rake autospec`.

It works much like the `parallel_tests` gem (and relies on it), but
makes each subprocess use a machine-readable formatter and parses this
output in order to provide a better overall summary.

(It's called interleaved, because parallel was taken and naming is
hard).

* Make popen3 invocation safer

* Use FileUtils instead of shelling out

* DRY up reporter

* Moved summary logic into Reporter

* s/interleaved/turbo/g

* Move Reporter into its own file

* Moved run into its own class

* Moved Runner into its own file

* Move JsonRowsFormatter under TurboTests

* Join on threads at the end

* Acted on feedback from eviltrout
2019-06-21 10:59:01 +10:00
Sam Saffron
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Sam Saffron
12e78d8915 DEV: add wrapper for rspec running in docker 2019-05-05 13:43:45 +10:00
Sam Saffron
3b77fb1fb1 DEV: support symlinked plugins in bin/rake autospec
Previously autospec would not pick up save if you saved a plugin in a
symlinked path, this broke quite a few workflows

We now maintain a reverse map so we can correctly re-run specs in plugins
2019-05-03 13:51:07 +10:00
Sam
0e6d0b0f8e FEATURE: always update docker image for dev prior to launching
Previously people could stay stuck on old dev images, this ensures they
are always on latest when booting
2019-02-18 13:12:00 +11:00
Sam
384135845b FEATURE: introduce ultra_low priority queue
This commit introduces an ultra low priority queue for post rebakes. This
way rebakes can never interfere with regular sidekiq processing for cases
where we perform a large scale rebake.

Additionally it allows Post.rebake_old to be run with rate_limiter: false
to avoid triggering the limiter when rebaking. This is handy for cases
where you want to just force the full rebake and not wait for it to trickle
2019-01-17 14:53:19 +11:00
Sam
77d947701c DEV: Add missing discourse script to docker dev 2019-01-07 13:10:37 +11:00
Sam
fef45789c4 DEV: improve docker dev environment
- pass USER=discourse which is not passed in from docker exec (improves backup restore)

- add unicorn script and unicorn support (expose port 9292)
2018-12-31 10:45:08 +11:00
Sam
6cf2e64e44 DEV: always run migrations for plugins in dev
Otherwise devs forget to run plugin migrations and get confused when test
does not work. We ship critical migrations in the bundled polls plugin.
2018-12-06 17:19:46 +11:00
Rishabh
10723ba1c2
Remove Docker dev-install instructions
To have a single source of truth for these install instructions in here:
https://meta.discourse.org/t/beginners-guide-to-install-discourse-for-development-using-docker/102009
2018-11-16 15:17:21 +05:30
Sam
f9f490e957 DEV: bin/unicorn -x to run without sidekiq 2018-08-17 10:01:44 +10:00
Sam
b8667c77c4 DEV: adjust bin/unicorn to support -p properly 2018-08-15 10:35:24 +10:00
Sam
c8d369a659 DEV: support -p in rails s
if you need to specify port
2018-08-15 10:25:00 +10:00
Sam
b504398a91 DEV: keep bin/unicorn clean from log noise 2018-08-14 08:49:57 +10:00
Robin Ward
cc90ed3870 Don't look for the only argument, but the first one 2018-08-09 10:14:45 -04:00
Guo Xiang Tan
56d3e07206 Use string when setting env value. 2018-08-07 16:38:31 +08:00
Guo Xiang Tan
81c6849194 Default rails server port should be 3000. 2018-08-07 16:33:04 +08:00
Sam
84c4dfd22d correct issues 2018-08-07 18:29:41 +10:00
Guo Xiang Tan
9e63ebdf0f DEV: Allow UNICORN_PORT env to be set. 2018-08-07 16:20:08 +08:00
Sam
f3549291a3 DEV: use unicorn in development
This commit also cleans up a bunch of pointless noise each time we boot app

- narrative was loading i18n cause redefinition of consts
- discourse.rb was loaded twice as was auth
- bin/unicorn now does all the smart things and boots unicron in dev
- bin/rails s will boot unicorn with no params
- remove bin/puma which only causes confusion
2018-08-07 17:13:47 +10:00
mmayoNR
a900c9bf93 pass env vars into local dev image (#6212) 2018-08-01 16:44:27 +10:00
Kyle Zhao
3e7638e3f5 Improve docker performance with delegated mount flag (#5760) 2018-04-16 10:56:35 +02:00
Jay Pfaffman
2658ef5e0b FIX: reset_db (#5617)
To get the database rebuilt requires a restart of the container. 

I also added a rake admin:create, since it's useful, and it's what `boot_dev --init `does.
2018-03-05 17:30:08 -05:00
Sam
32d881399f avoid getting duplicates in docker dev paths 2017-12-15 10:11:40 +11:00
Sam
e0f660d9a9 lets crash out on error 2017-12-14 18:03:17 +11:00
Guo Xiang Tan
77d4c4d8dc Fix all the errors to get our tests green on Rails 5.1. 2017-09-25 13:48:58 +08:00
Guo Xiang Tan
5012d46cbd Add rubocop to our build. (#5004) 2017-07-28 10:20:09 +09:00
Sam
234694b50f Feature: CommonMark support
This adds the markdown.it engine to Discourse.
https://github.com/markdown-it/markdown-it

As the migration is going to take a while the new engine is default
disabled. To enable it you must change the hidden site setting:
enable_experimental_markdown_it.

This commit is a squash of many other commits, it also includes some
improvements to autospec (ability to run plugins), and a dev dependency
on the og gem for html normalization.
2017-06-23 12:01:33 -04:00
Sam
4c1b0e64d7 Merge pull request #4874 from davidtaylorhq/docker_dev_rake
Use `bundle exec` for docker_dev rake
2017-05-24 11:51:13 -04:00
Sam
e7c2ad41ca Move discourse dev data out of tmp
Fix watch for restart so it works with puma
2017-05-18 11:36:24 -04:00
David Taylor
058cde1fc5 Use bundle exec for docker_dev rake 2017-05-16 17:18:45 +01:00
Rafael dos Santos Silva
2e152f4d39 FIX: Use latest docker image for dev 2017-05-16 12:05:11 -03:00
Sam
7e15077bec add puma binstub 2017-05-15 09:08:51 -04:00
Sam Saffron
9051ca7959 correct sample to work with rails.vim 2017-04-25 09:51:49 -07:00
Sam Saffron
f04fbf911a FEATURE: in vim dev you can focus on spec line in autospec
instructions in bin/notify_file_change
2017-04-25 09:13:29 -07:00
Sam
3906f55132 add binstub for rspec 2017-04-17 12:58:34 -04:00
ckeboss
6bfe92e19e Update boot_dev 2017-04-12 10:33:56 -07:00
ckeboss
513f1f065a Wrap directory names in quotes
When there are spaces in any of the directories referenced, bash will error out. This wraps those items in quotes to allow bash to parse the path names properly.
2017-04-12 09:41:23 -07:00
Leo McArdle
a11c4a8078 FIX: start sidekiq process for all queues in docker dev
https://meta.discourse.org/t/docker-dev-only-starting-critical-sidekiq-queue/58242
2017-03-02 18:00:24 +00:00
Sawood Alam
911629735c
Removed the fallback mailcatcher install command 2016-12-13 09:26:11 -05:00
Sam
b650a45b37 tweak perf of rake and rails 2016-12-13 14:51:07 +11:00
Sam
4faec8af96 add some extra helper scripts 2016-12-13 09:24:31 +11:00
Sam
97e2a614c8 add wrappers for mailcatcher and sidekiq
add symlink to simplify invoking docker commands `d/rake`
2016-12-13 09:05:45 +11:00
Sam
107a21c069 update readme for docker dev 2016-12-12 18:00:25 +11:00
Sam
4b0f013edf migrate test as well 2016-12-12 17:47:24 +11:00
Sam
1cbb8dc9b7 improve dev process 2016-12-12 17:39:20 +11:00
Jared Reisinger
b7ea3de7bb Switch from hard-coded image version to 'latest' 2016-09-22 12:58:43 -07:00
Jared Reisinger
4db3f53c87 Improve the "develop inside Docker" experience
This PR is intended to work in concert with the discourse_docker changes
in https://github.com/discourse/discourse_docker/pull/292.  I have used
those changes to build a local "discourse_dev" image, and then
_**these**_ changes to easily spin up and work on Discourse and plugin
functionality.  It's working well for me, but of course YMMV.

Add `--init` and `--patch` options to bin/docker/boot_dev to make it
easier to spin up a container running Discourse out of a development
directory.

Add `bin/docker/README.md` to explain how to go about using the docker
command-line tools.

Tweak the Docker command-line tools to use the `-u` option to specify
user rather than changing to the user "inside" the container via `chpst`
(This way, we don't have to explicitly specify the HOME environment
variable, either.)

Add `bin/docker/shell` command to make it easy to jump inside the
running container.
2016-09-20 15:35:50 -07:00
Sam
2c4f290786 docker dev binaries 2015-11-02 13:33:08 +11:00
Sam Saffron
b2c30d0e34 Revert "travis likes old bundler"
This reverts commit 1adda31cbc.
2015-04-16 16:15:04 +10:00
Sam Saffron
1adda31cbc travis likes old bundler 2015-04-16 16:13:48 +10:00
David Celis
49bc7692bf Include binstubs generated by Rails 4
Commit b516ecc added `bin/` to the .gitignore file. Now that Discourse
runs using Rails 4 by default, however, we should include the binstubs
generated by `rake rails:update:bin` in version control as this is the
recommendation of the Rails core team. Additionally, for those wishing
to deploy Discourse to Heroku, these binstubs are actually mandatory
according to [this article](https://devcenter.heroku.com/articles/rails4).

Other binstubs can continue to be ignored.

Signed-off-by: David Celis <me@davidcel.is>
2014-02-06 11:15:00 -08:00