Commit Graph

90 Commits

Author SHA1 Message Date
David Taylor
2955d64703
DEV: Allow annotations to work in symlinked plugins, add binstub (#13635) 2021-07-05 15:43:10 +01:00
Penar Musaraj
fbe004cc63
DEV: Do not add proxy argument when running ember-cli test (#13498) 2021-06-23 10:03:52 -04:00
David Taylor
3b6d6c7024
DEV: Set DISCOURSE_PORT when spawning unicorn via ember-cli -u (#13346)
This means that Discourse will use the ember-cli proxy's port number in various places like auth redirects and emails
2021-06-09 15:32:28 +01:00
Penar Musaraj
513bfc3a6c
DEV: bin/ember-cli standalone by default (#13344) 2021-06-09 09:48:43 -04:00
Sam
0241748876
DEV: ember-cli -u can be used to run a standalone dev discourse (#13336)
Previously we would need to launch unicorn separately this achieves
the same goal by making 2 modifications:

1. If -u is supplied ember-cli binary will launch and monitor ember cli and unicorn
2. We suppress 200 requests to keep console clean (we may consider moving to development rails logs)


Also cleans out output a bit by supplying silent flags to yarn.
2021-06-09 12:44:33 +10:00
David Taylor
78f9d47ab1 DEV: Add non-x86_64 warning to d/boot_dev
Running a development environment using Docker's qemu architecture emulation is currently not possible because `inotify` is not supported: https://github.com/docker/for-mac/issues/5321
2021-05-21 16:51:10 +01:00
David Taylor
73050d9049 DEV: Update docker development binstubs
- Add `d/ember-cli`, and publish port 4200
- Remove `d/sidekiq`. Sidekiq is now started with the rails server
- Move all `docker exec` logic into a single place, so we have one place to set environment variable pass-throughs
- Use `exec` for all bash scripts, so that return statuses are passed back correctly
- Avoid using `bin/bash -c` unnecessarily, because it makes escaping arguments difficult
2021-05-21 16:51:10 +01:00
David Taylor
ab23412113 DEV: Only mount top-level plugin symlinks
Sometimes plugins directories will end up with other symlinks (e.g. inside node_modules folders). This logic does not work with deeply nested symlinks, and they are unlikely to be necessary for the plugin to work. Therefore we should only look for symlinks in the top-level of the `plugins` directory
2021-05-21 16:51:10 +01:00
Robin Ward
f165f98cf7
DEV: Run yarn install when running bin/ember-cli (#13102)
Some people have noticed that if we change the packages in package.json
that they have to manually run `yarn install` or Discourse won't work.

This adds `yarn install` to the `bin/ember-cli` helper we run. It seems
quite fast if there is nothing to install so it shouldn't hurt to do
this every time we start the server.
2021-05-20 14:30:22 -04:00
Gerhard Schlager
4c26dd09e4
DEV: Update bin/bundle (#13067)
This fixes the following error I've been seeing lately in RubyMine:

> Error:Your `bin/bundle` was not generated by Bundler, so this binstub cannot run.
> Replace `bin/bundle` by running `bundle binstubs bundler --force`, then run this command again.
2021-05-14 19:54:10 +02:00
David Taylor
b6b27bc383
DEV: Improve auto-restart parent process detection logic (#13068)
The auto restart logic was sending a USR2 to the parent process without checking what the parent process actually was. In some situations, it might not be the `bin/unicorn` supervisor.

This commit switches to use a global variable for the supervisor PID. This will be much less prone to unexpected behavior.
2021-05-14 18:17:31 +01:00
Jeff Wong
7a0e5c8cb9
FIX: properly filter ember-cli args (#13011)
Was previously rejecting all args.
2021-05-10 10:39:54 -07:00
Penar Musaraj
49de5dadf7
DEV: Ember CLI postinstall hook and shortcut (#12987) 2021-05-10 09:34:24 -04:00
Jarek Radosz
53c06442ef
DEV: Clean up .gitignore (#12981)
(inspired by martin-brennan's recent adventure with `tags` exclusion)

**tl;dr: some of it is obsolete, some is dev-env specific and should live in *your* ~/.gitignore.**

---

To enable global gitignore (`~/.gitignore`) run:
```bash
git config --global core.excludesFile '~/.gitignore'
```

Then create that file and add your VIM/VSCode/Sublime/Emacs/Eclipse/RubyMine/JetBrains/macOS/Arch/Windows 95-specific stuff.

---

Reasons for removal:

* `bin` - never really ignored, all the files are checked in
* `.DS_Store`, `._.DS_Store` - OS specific
* `.sass-cache/*`, `/.bundle`, `/bundle/*`, `/cache`, `/logfile`, `!/plugins/discourse-nginx-performance-report`, `MiniProfiler/Ruby/rack-mini-profiler-2.0.1a.gem`, `config/fog_credentials.yml`, `/public/stylesheet-cache/*`, `script/download_db`, `script/refresh_db`, `config/version.rb` - no longer used?
* `/log/*.log` - covered by /log
* `bootsnap-load-path-cache`, `bootsnap-compile-cache` - bootsnap now keeps its ~~trash~~ temporary files in `/tmp`
* `/.project`, `/.buildpath`, `/.byebug_history`, `/.idea`, `discourse.sublime-workspace`, `*~`, `*.swp`, `*.swo`, `*.swm`, `config/multisite1.yml`, `.rvmrc`, `.ruby-version`, `.ruby-gemset`, `.rbenv`, `bundler_stubs/*`, `*.db`, `*.iml`, `*.swn`, `/package-lock.json`, `.vscode`, `.envrc`, `tags` - dev-env specific, see the note at the top of the commit
2021-05-10 13:43:13 +02:00
Jeff Wong
bc3c3d56e0
DEV: do not cd into yarn dir, use --cwd argument to yarn instead (#12986)
* DEV: do not cd into yarn dir, use --cwd argument to yarn instead

Co-authored-by: Robin Ward <robin.ward@gmail.com>
2021-05-07 09:58:57 -07:00
David Taylor
ac1d344368
DEV: Update server startup warning to be more specific (#12949)
Using `bundle exec` will slow down server startup by at least 0.5s. `bin/unicorn` has built-in handling of bundler dependencies, so it is better to launch `bin/rails s` or `bin/unicorn` directly.
2021-05-05 13:33:11 +01:00
David Taylor
0188d53f3a
DEV: Add performance warning message when running rails s (#12912)
In Discourse, `rails s` ultimately launches the `bin/unicorn` script. However, the overhead of `rails` launching `bin/rails`, and then in turn `bin/unicorn` can be non-trivial. Therefore it is much better to run `bin/unicorn` directly.

This commit prints a warning message to STDERR when `rails s` is used. Functionality is unchanged.
2021-04-30 11:50:06 +01:00
Robin Ward
51f872f13a
DEV: Require Ember CLI to be used in development mode (#12738)
We really want to encourage all developers to use Ember CLI for local
development and testing. This will display an error page if they are not
with instructions on how to start the local server.

To disable it, you can set `NO_EMBER_CLI=1` as an ENV variable
2021-04-29 14:13:36 -04:00
Sam
307cb1d114
DEV: add helper script to run ember cli (#12005)
This little helper script allows for easy ember cli development.

To see the options run `bin/ember-cli -h`

It allows you to proxy try.discourse.org with the `bin/ember-cli --try`
switch, which effectively allows for some development without a rails installed.

It passes on arguments to ember-cli so you can customize port and so on.

It makes the assumption that on local people are using `bin/unicorn` for
development. (it includes some extra discourse specific helpers)
2021-02-09 16:33:14 +11:00
Arpit Jalan
bbf6cc4f18
DEV: require fileutils gem on boot (#11542) 2020-12-21 16:31:06 +11:00
Robin Ward
2eb9c0f3dd FIX: notify_file_change was outputting a command to vim
The check for `socat` needs to be silenced too, or vim will output the
path to `socat` every time you save a file.
2020-12-10 10:03:57 -05:00
Alessandro Bertacco
dc2f5064e2
DEV: support for relative symlinks for plugins in docker dev (#10955)
These changes gives the possibility to use relative symlinks for plugins. Previously only absolute symlinks was supported.
2020-10-21 09:39:53 +11:00
David Taylor
46d1c91e1a
DEV: Make notify_file_change fallback to nc when socat missing (#10903) 2020-10-13 17:03:24 +01:00
Kane York
4635be10c8 DEV: Add docker cleanup script to d/ folder 2020-03-01 12:09:07 -08:00
Sam Saffron
d8412f409a DEV: resolve symlinks in docker dev
symlinks where not fully resolved leading to docker not booting when you
had symlinks in the plugins directory pointing at relative paths.
2020-01-13 10:33:34 +11:00
Martin Brennan
aa2b3e71b4 Minor refactor of bin/unicorn cache clean
* move git ls-files for plugins folder into one line
2019-11-20 08:47:51 +10:00
Sam Saffron
9652fdd111 DEV: if tmp is missing create it
Some scripts and people may be nuking the entire tmp folder
2019-11-20 09:29:34 +11:00
Martin Brennan
93d7abe372
FIX: Automatically recover from bad sprockets cache in development (#8364)
We were having issues in development mode where the JS code had errors due to a bad cache. When starting a server in development mode in bin/unicorn we now get the git sha of the discourse HEAD and get a git sha of all plugins, and store them in a file. If the sha has changed then we delete tmp/cache to refresh the assets cache.
2019-11-19 09:15:09 +10:00
Sam Saffron
ff33899323 FEATURE: allow publishing docker dev either locally or net wide
Previously we had no control over how internal ports in the containter got
published.

Following UNICORN_BIND_ALL=true setting this broke docker dev env and exposed
this weakness.

The new `d/boot_dev` will only export on localhost, if you wish to export
network with use `d/boot_dev -p`
2019-11-04 12:51:35 +11:00
Mark VanLandingham
9b4aba0d39
DEV: support --fail-fast in bin/turbo_rspec (#8170)
* [WIP] - default turbo spec env to test

* FEATURE: support for --fast-fail in bin/turbo_rspec

* fast-fail -> fail_fast to match rspec

* Moved thread killing outside of fail-fast check

* Removed failure_count incrementation from fast_fail_met
2019-10-09 09:40:06 -05:00
Sam Saffron
7c084c7cab DEV: allow USR2 to restart unicorn in dev
Well all this does is amends a commit comment cause I was over eager and
pushed previous commit.

This is the comment I wanted....

This allows `pkill -USR2 -f 'ruby bin/unicorn'` to restart current unicorn

It is handy if you want to bind a keyboard shortcut for unicorn restarts
in dev.

By doing so you can avoid finding the terminal, hitting ctrl-c and then
hitting up, enter, heading back to browser.

It saves time.

Automate stuff, you will not regret it...
2019-09-11 17:45:47 +10:00
Sam Saffron
fb8d1f35a4 DEV: support USR2 to restart unicorn in dev mode
In dev mode sending USR2 to the unicorn master supervisor process will
restart unicorn.

This allows a simple script like this to restart unicorn in dev:

```
#!/usr/bin/env bash

kill -s USR2 `ps aux | grep ruby\ bin\/unicorn | grep -v grep | awk '{print $2}'`
```
2019-09-11 17:35:58 +10:00
hawm
16681cb648 Fix docker image name (#8058) 2019-09-04 16:19:48 +10:00
Kyle Zhao
2fbafd077c DEV: Mount plugin symlinks to dev docker container (#8002)
When developing using docker, in order to support symlinks in the
`plugins/` directory, this reads the symlinks' values and mounts them to
the dev docker container.
2019-08-14 18:13:01 +10:00
Kyle Zhao
e2df331fdb DEV: default to test env when running d/rspec (#7962) 2019-08-02 16:40:46 +10:00
Daniel Waterworth
c3db5925a8 FIX: Turbo tests exit codes 2019-07-09 08:51:23 +01:00
Daniel Waterworth
d6aa92e98e DEV: Add a verbose option to ./bin/turbo_rspec 2019-06-27 15:49:21 +01:00
Sam Saffron
fc84e23b71 DEV: allow bin/turbo_tests to run tests without params 2019-06-21 11:33:22 +10:00
Daniel Waterworth
e18ce56f4b DEV: Add a new way to run specs in parallel with better output (#7778)
* DEV: Add a new way to run specs in parallel with better output

This commit:

 1. adds a new executable, `bin/interleaved_rspec` which works much like
    `rspec`, but runs the tests in parallel.

 2. adds a rake task, `rake interleaved:spec` which runs the whole test
    suite.

 3. makes autospec use this new wrapper by default. You can disable this
    by running `PARALLEL_SPEC=0 rake autospec`.

It works much like the `parallel_tests` gem (and relies on it), but
makes each subprocess use a machine-readable formatter and parses this
output in order to provide a better overall summary.

(It's called interleaved, because parallel was taken and naming is
hard).

* Make popen3 invocation safer

* Use FileUtils instead of shelling out

* DRY up reporter

* Moved summary logic into Reporter

* s/interleaved/turbo/g

* Move Reporter into its own file

* Moved run into its own class

* Moved Runner into its own file

* Move JsonRowsFormatter under TurboTests

* Join on threads at the end

* Acted on feedback from eviltrout
2019-06-21 10:59:01 +10:00
Sam Saffron
30990006a9 DEV: enable frozen string literal on all files
This reduces chances of errors where consumers of strings mutate inputs
and reduces memory usage of the app.

Test suite passes now, but there may be some stuff left, so we will run
a few sites on a branch prior to merging
2019-05-13 09:31:32 +08:00
Sam Saffron
12e78d8915 DEV: add wrapper for rspec running in docker 2019-05-05 13:43:45 +10:00
Sam Saffron
3b77fb1fb1 DEV: support symlinked plugins in bin/rake autospec
Previously autospec would not pick up save if you saved a plugin in a
symlinked path, this broke quite a few workflows

We now maintain a reverse map so we can correctly re-run specs in plugins
2019-05-03 13:51:07 +10:00
Sam
0e6d0b0f8e FEATURE: always update docker image for dev prior to launching
Previously people could stay stuck on old dev images, this ensures they
are always on latest when booting
2019-02-18 13:12:00 +11:00
Sam
384135845b FEATURE: introduce ultra_low priority queue
This commit introduces an ultra low priority queue for post rebakes. This
way rebakes can never interfere with regular sidekiq processing for cases
where we perform a large scale rebake.

Additionally it allows Post.rebake_old to be run with rate_limiter: false
to avoid triggering the limiter when rebaking. This is handy for cases
where you want to just force the full rebake and not wait for it to trickle
2019-01-17 14:53:19 +11:00
Sam
77d947701c DEV: Add missing discourse script to docker dev 2019-01-07 13:10:37 +11:00
Sam
fef45789c4 DEV: improve docker dev environment
- pass USER=discourse which is not passed in from docker exec (improves backup restore)

- add unicorn script and unicorn support (expose port 9292)
2018-12-31 10:45:08 +11:00
Sam
6cf2e64e44 DEV: always run migrations for plugins in dev
Otherwise devs forget to run plugin migrations and get confused when test
does not work. We ship critical migrations in the bundled polls plugin.
2018-12-06 17:19:46 +11:00
Rishabh
10723ba1c2
Remove Docker dev-install instructions
To have a single source of truth for these install instructions in here:
https://meta.discourse.org/t/beginners-guide-to-install-discourse-for-development-using-docker/102009
2018-11-16 15:17:21 +05:30
Sam
f9f490e957 DEV: bin/unicorn -x to run without sidekiq 2018-08-17 10:01:44 +10:00
Sam
b8667c77c4 DEV: adjust bin/unicorn to support -p properly 2018-08-15 10:35:24 +10:00