Is generally aligned with out SAML2 group sync functionality, but for
OIDC based upon feedback in #3004.
Neeeded the tangental addition of being able to define custom scopes on
the initial auth request as some systems use this to provide additional
id token claims such as groups.
Includes tests to cover.
Tested live using Okta.
For PR #3406
- Updated naming from 'redirect' to 'initate/initation'.
- Updated phpunit.xml and .env.example.complete files with the new
option.
- Cleaned up controller logic a bit.
- Added content and design to the new initation view to not leave user
on a blank view for a while.
- Added non-JS button to initiation view as fallback option for
progression.
- Moved new test to it's own Test class and expanded with additional
scenario tests for better functionality coverage.
Linebreaks would previously essentially be removed during index and
hence joined to adjacent words, breaking prefix matching.
Added test to cover.
For #3508
- Updates book/shelf cover image handling for easier cloning/handling.
- Adds core logic for promoting books/chapters up a level.
- Enables usage of book/shelf cover image via API.
Related to #1087
Moved export templates elements into their own folder for better
grouping of logical usage.
Within the base export template, added some body classes to allow easier
targeted customisation via custom head css.
Split content of export templates into smaller partials for easier
future customization.
Closes#3443
Was reporting wrong due to use of old polymorphic namespace references.
Test was not picking up as assertElementContains had wider scope than
expected, looking within the HTML of the element instead of the text
which you might expect. Updated test helper to look at text instead.
Dumping details that were binary, such as the jpegphoto data, would
cause the dump to fail on the encoding to JSON.
This change forces content to be UTF8 before dumping.
Updated existing test to cover.
Closes#3396
Made the following changes, many of these are just to align with
existing conventions.
- Updated urls to be hypenated, instead of underscored, to match other system endpoints.
- Updated URL parameter to be `deletionId` instead of `id`, and removed the ID-based comment on controller methods, so the required ID model is clear from the URL alone, since its not clear from the URL endpoint alone like existing endpoints. This follows the pattern used in the "web" routes.
- Added extra detail on some controller method comments, and copied permission comment to each method.
- Removed existing field visibility mechanisms to use simpler model-based visibility since we didn't need anything too special here (After some of my other changes).
- Allowed the "deletable" model to be shown in response to provide a little more detail on the main deleted item.
- Updated parent/child-count loading to be on the "deletable" model instead of additional properties which results in simpler controller logic and enforces the idea these are relations on the deletable, not the deletion itself. It also removes additional exposure of model namespacing.
- Updated (int) casts to intval, just since that's our most common conversion method in the codebase.
- Testing: Removed `actingAsAuthorizedUser` and used the admin user instead to prevent extra auth steps on each test.
- Testing: Cut logic/data-checks from tests if already covered by other tests.
- Testing: Added simple assertions for delete/restore response data.
- Examples: Updated list example to reflect changes.
Review of PR #3377
To be followed up with changes to polymorphic relations to hide
namespacing.
Decided it's relevant to entity updated_at since tags are now indexed
alongside content.
- Also fixed tags not applied on shelf.
- Also enforced proper page API update validation.
- Adds tests to cover.
For #3319Fixes#3370
- Added testing check to buffer stop/clear on streaming output due to
interference during tests.
- Made content-disposition header a little safer in download responses.
- Also aligned how we check for testing environment.
- Fixed issue where redirect for `/settings` view would not be ran
through base url generator so would not create a correct path in some
cases. Now routed through controller with normal redirect.
- Fixed custom head content being active on settings pages due to route
name changes, for when viewing settings, in last release.
Fixes#3356 and #3355
- Updated PhpStan PHP version option to match project.
- Applied StyleCI changes.
- Updated static to self in WebhookFormatter, following static analysis
guidance.
- Fixed mis-matched header tags.
- Split settings out to new views using a core shared layout.
- Extracted added language text to translation files.
- Updated settings routes to be dynamic to category.
- Added redirect for old primary settings route.
- Updated existing tests to cover settings route changes.
- Added tests to cover settings view.
- Improved contrast of settings links for dark mode.
Creates a new organsied formatting system for webhook data, with
interfaces for extending with custom model formatting rules.
Allows easy usage & extension of the default bookstack formatting
behaviour when customizing webhook events via theme system, and keeps
default data customizations organised.
This also makes the following webhook data changes:
- owned_by/created_by/updated_by user details are loaded for events with
Entity details. (POTENTIALLY BREAKING CHANGE).
- current_revision details are loaded for page update/create events.
Added testing to cover added model formatting rules.
For #3279 and #3218
- Configurable via 'ALLOWED_IFRAME_SOURCES' .env option.
- Also updated how CSP rules are set, with a single header being used
instead of many.
- Also applied CSP rules to HTML export outputs.
- Updated tests to cover.
For #3314
- For some reason, TinyMCE would handle empty paragraphs with a ' '
by default but this would be removed when the paragraph had an
attribute. This was fine in the old editor.
- This changes the approach to use '<br>' tags within elements
for "spaced emptiness".
- For compatbility with any existing empty paragraphs, I updated the
styles to show default height for empty paragraph sections.
- This also makes changes to help preserve encoded html tags
since they were getting converted along the journey.
Related to #3302
Fixes issue where certain errors would not show to the user
due to extra navigation jumps which lost the error message
in the process.
This simplifies and aligns exceptions with more directly
handled exception usage at the controller level.
Fixes#3264
- Removed old 'editor-*-update' commands to instead use the aligned
'editor::replace' command that we already have.
- Changed the way custom styles are loaded for the WYSIWYG editor so we
don't need an API call but instead scape content from the parent page
header using comments as identifiers. Added tests to ensure comments
exist and align.
- To display license info along with shortcuts.
- Extracted out plain layout from 503 error page.
- Added tests to ensure license references are as expected.
- Required extracting logic into repo.
- Changed some existing creation paths to standardise behaviour.
- Added test to cover new endpoint.
- Added extra test for user delete to test migration.
- Changed how permission errors are thrown to ensure the right status
code can be reported when handled in API.
- Required changing the docs generator to handle more complex
object-style rules. Bit of a hack for some types (password).
- Extracted core update logic to repo for sharing with API.
- Moved user update language string to align with activity/logging
system.
- Added tests to cover.
- Updated routes to use new format.
- Changed how hidden fields are exposed to be more flexible to different
use-cases.
- Updated properties available on read/list results.
- Started adding testing coverage.
- Removed old unused UserRepo 'getAllUsers' function.
Related to #2701, Progression of #2734
- Added tests to cover.
- Refactored some existing testing.
- Requires password or external_auth_id to be provided. Defaults to
password.
- Randomly sets password to 32 digit random chars if external_auth_id
provided instead.
For #3222
Custom homepage usage will now be checked before any actioning
of deletion rather than potentially causing an exception acting
during the deletion.
Previously a deletion could still be created, within the recycle bin,
for the parent which may lead to the page being deleted anyway.
For #3150
- Aligned permissions control with move operations to check
delete/create permissions against old/new locations.
- Added tests to cover additional permissions scenarios.
Previous implemenations were hard to read so changing to be more
logically simplistic. Still needs further coverage in tests and
review/alignment of permissions to use.
Adds apng sniffing when generating thumbnails with retained ratios to
serve the original image files, as we do for GIF images, to prevent
the image being resized to a static version.
Is more tricky than GIF since apng file mimes and extensions
are the same as png, we have to detect part of the file header
to sniff the type. Means we have to sniff at a later stage
than GIF since we have to load the image file data.
Made some changes to the image thubmnail caching while doing
this work to fit in with this handling.
Added test to cover.
For #3136.
- Added a user-configurable timeout option to webhooks.
- Added webhook fields for last-call/error datetime, in addition to last
error string, which are shown on webhook edit view.
Related to #3122
- Copies via loading in model on create view.
- Updated role views while editing to bring up to similar format as
that used for more modern app areas.
- Added tests to cover.
Related to #1123
Altered access & usage of the /search/users/select endpoint with the
following changes:
- Removed searching of email address to prevent email detail discovery
via hunting via search queries.
- Required the user to be logged in and have permission to manage users
or manage permissions on items in some way.
- Removed the user migration option on user delete unless they have
permission to manage users.
For #3108
Reported in https://huntr.dev/bounties/135f2d7d-ab0b-4351-99b9-889efac46fca/
Reported by @haxatron
- Renamed to "ActivityLogger" to be more focused in usage.
- Extracted out query elements to seperate "ActivityQueries" class.
- Removed old 'addForEntity' activity method to limit activity record
points.
Allows easy direct linking and usage of the HTML preview content
we show in the UI when viewing search results.
Note: preview_html content is a rough representation only, it does not
match exactly what was matched in the database-search-operation which
finds the results.
For #3096 and #3080
Page-related items added on drafts could be visible in certain scenarios
since the applied permissions query filters would not consider
page draft visibility.
This commit alters queries on related items to apply such filtering.
Included test to cover API scenario.
Thanks to @haxatron for reporting.
- Replaced iframe elements with anchor elements wrapped in a paragraph.
- Extracted PDF generation action to seperate class for easier mocking
within testing.
- Added test to cover.
For #3077
Old command registration method was interfering with default commands,
causing only a limited subset of commands to show overall.
This change follows the method the frameworks uses when loading in from a
directory to prevent issues with run/load order.
Changed how the command registration was handled due to complications of
action order found during testing. Now the theme service will resolve
and directly register the command on the Kernel instead of them being
fetched from the ThemeService from within Kernel.
More direct, Seems to work.
Updated the back button to be a proper link instead of a reference to
the last viewed URL since it could break if the last page was the
current one (On validation for example).
Includes test to cover.
Also applied some styleCI changes.
Fixes#2834
Was done for convenience but could potentially be exploited by an
attacker using signing up via one of these routes, then forwarding
an email confirmation to another user so they unknowingly utilise
an account someone else controls.
Tweaks the flow of confirming email, and the user invite flow.
For #3050
This change means that code blocks are now included still wrapped in
their pre tags, as we do for tables and lists.
Previously the <code> inner content would be included which would lead
to a generally bad/broken presentation.
Hopefully should not be a breaking change as section include tags for
code was tricky to get to, and it was in a semi-broken state.
For #2406
As per #3047.
Also made some SAML specific fixes:
- IDP initiated login was broken due to forced default session value.
Double checked against OneLogin lib docs that this reverted logic was fine.
- Changed how the saml login flow works to use 'withoutMiddleware' on
the route instead of hacking out the session driver. This was due to
the array driver (previously used for the hack) no longer being
considered non-persistent.
Is a little awkward, emulates a 'list' API endpoint but has unstable
paging and does not support filters/sort. This is detailed on the
endpoint though.
Made some updates to the docs system to better support parameters
and examples on GET requests.
Includes tests to cover.
For #909
Removing prefix route groups out of visual preference.
Those don't really save much and I prefer seeing the complete
paths when going down the list to better guage where I am.
Updating filters via sidebar would wipe the created_by/update_by filters
since these were not part of the sidebar filter form.
This adds them, if existing, as hidden inputs.
Includes tests to cover.
Closes#2736
Markdown content was not being stored, only the sent
HTML representation, causing the draft to show as HTML upon next edit.
Added test to cover.
Fixes#3054
This adds parsing of page content so that headers apply a boost to
scores in the search term index.
Additionally, this merges title and content terms to reduce the amount
of stored terms a little.
Includes testing to cover.
Session was being lost due to the callback POST request cookies
not being provided due to samesite=lax. This instead adds an additional
hop in the flow to route the request via a GET request so the session is
retained. SAML POST data is stored encrypted in cache via a unique ID
then pulled out straight afterwards, and restored into POST for the SAML
toolkit to validate.
Updated testing to cover.
Relevant to comments in 7224fbcc89.
Added test cases. Ensured they failed pre-commit.
Also tested a range of the altered endpoints manually on both local and
s3-like filesystems.
- Prevents authenticated responses being visible when back button
pressed in browser.
- Previously, 'no-cache, private' was added by default by Symfony which
would have prevents proxy cache issues but this adds no-store and a
max-age option to also invalidate all caching.
Thanks to @haxatron via huntr.dev
Ref: https://huntr.dev/bounties/6cda9df9-4987-4e1c-b48f-855b6901ef53/
- Added testing to cover warning cases.
- Refactored logic to be simpler and move much of the business out of
the controller.
- Added new message that's more suitable to the case this was handling.
- For detecting an outdated draft, checked the draft created_at time
instead of updated_at to better fit the scenario being checked.
- Updated some method types to align with those potentially being used
in the logic of the code.
- Added a cache of shown messages on the front-end to prevent them
re-showing on every save during the session, even if dismissed.
Review of #2936
- Added testing to cover
- Added APP_PROXIES to .env.example.complete with details.
- Renamed migration to better align the name and to set the migration
date to fit with production deploy order.
- Removed index from IP column in migration since an index does not yet
provide any value.
- Updated table header text label.
- Prevented IP recording when in demo mode.
Done a little reorganisation while there of misplaced tests.
Moved MarkdownTest to a new PageEditorTest to avoid confusion with
other markdown elements and to align with other page tests.
Was causing things like emjoi within script content to be somewhat
mangled. Instead we force UTF8 only parsing via XML declaration.
Added test to cover.
For #2923
Fixes a strange issue of HTML tags within script tags being malformed
when part of the HTML custom head content due to the PHP parsing we do.
DOMDocument seemed to cause this upon load.
Adding LIBXML_SCHEMA_CREATE to the ->loadHTML call seems to fix this but
not really sure why. Doesn't seem to cause further issues though.
Tested with multiple scripts and styles and comments and meta tags.
- Also added new testing class to cover.
- As part of testing, added new folder within tests to house setting
specific tests.
For #2914
In particular, That around the casing of dangerous values within
attributes. This uses some xpath translation to handle different casing
in contains searching.
WKHTMLtoPDF provides limited control for external fetching
so that will now be disabled by default unless
ALLOW_UNTRUSTED_SERVER_FETCHING=true is specifically set.
This new option will also control DOMPDF fetching.
During writing of the update notes, found that the upgrade path would be
tricky from a security point of view. If people were pending email
confirmation but had an active session, they could technically be
actively logged in after the next release.
Added middlware as an extra precaution for now.
- Updated auth system for mfa to not update intended URL so that the
user is not redirected to mfa setup after eventual login.
- Added notification for users setting up MFA, after setup when
redirected back to login screen to advise that MFA setup was complete
but they need to login again.
- Updated some bits of wording to display better.
- Restructured some of the route naming to be a little more consistent.
- Moved the routes about to be more logically in one place.
- Created a new middleware to handle the auth of people that should be
allowed access to mfa setup routes, since these could be used by
existing logged in users or by people needing to setup MFA on access.
- Added testing to cover MFA setup required flow.
- Added TTL and method tracking to session last-login tracking system.
Also added testing to cover as part of this in addition to adding the
core backup code handling required.
Also added the standardised translations for switching mfa mode and
adding testing for this switching.
- Added new DB column for control and role updated create/update actions.
- Created new middleware as a start to actual enforcement logic.
- Added indicator to role list of whether MFA is enforced.
On restore, added a link to the parent deletion restore if any exists
on a cascading parent. Added a test to cover this case to ensure its shown.
Also tweaked default empty state message on recycle bin item list to align
with new column count.
Also done a little existing code cleanup including a getUrl helper on
the deletion items.
Related to #2682 & #2594
For review of meta tag additions as per PR #2393.
This commit removes any image guesswork and only uses images that have
been set by the author for the specific content.
This also adds tests to cover the expected OG tags.
- Removed ZIP system for now, until the idea can be fleshed out.
- Added testing to cover.
- Upgraded used library.
- Added custom handling for BookStack callouts.
- Added HTML cleanup to better produce output for things like code
blocks.
Fixes issues thrown in custom HMTL head & page content filtering when
the content is comprised of only a single HTML comment.
Adds tests to cover.
For #2804
- Added test cases to cover.
- Altered parsing logic to be a little less reliant on regex.
- Added new iamge repo method for creating from data.
- Added extension validation and additional type support.
- Done some cleanup of common operations within PageContent.
- Added message to API docs/method to mention image usage.
For #2700 and #2631.
- Reduced options to single new configuration paramter instead of two.
- Moved more logic into UserAvatars class.
- Updated LDAP avatar import to also run on login when no image is
currently set.
- Added thumbnail fetching to search requests.
- Added testing to cover.
Related to PR #2320, and issue #1161
- Refactored some tag code bits while reviewing.
- Updated tag design in search listing to be more subtle.
- Moved tags out of entity-list-item-basic template and instead moved
them into entity-list-item, below the existing content.
- Tweaked existing tag colors a little.
- Changed tag icon to be more tag-like.
- Added tag-on-search test case.
Review of #2487, Related to #2462
Allows someone using the theme system to configure the social driver
before a redirect action occurs, by passing a callback as an additional
param to the theme 'addSocialDriver' method.
To ensure thhat the visibility is still set on local storage options
since the previous recent changes could cause problems where in
scenarios where the server user could not read images uploaded by the
php process user.
Closes#2758
- Also removed some old view service references.
- Updated TopFavourites query to be based on favourites table and join
in the views instead of the other way around, so that favourites still
show even if they have no views.
- Commands that run a truncate DB action failed due to messing up the
test transations so we mnaully work around that now to ensure a
transaction exists for the test to cleanup afterwards.
- Updated dompdf lib version
Also allowed config-centralised default user settings for this change
and bought existing user-level view options into that default settings
system to be cleaner in code usage.
For #2081
- Moved the ldap function out to our separate service for easier
testing.
- Added testing for the option.
- Moved tls_insecure part back up above connection start as found more
reliable there.
Done a lot of real-connection testing during this review.
Used wireshare to ensure TLS connection does take place.
Found LDAP_TLS_INSECURE=false can action unreliably, restarting php-fpm
helped.
Tested both trusted and untrusted certificates.
Made so footer link ordering, names and urls can be set.
Cleaned up some of the setting-service and added support for array
setting types, which are cleaned on entry and stored as json with a new
type indicator column on the settings table for auto-decode.
Also added testing to cover this feature.
Related to #1973 and #854
This permission was still checking based on created-by.
Updated testing to specifically check the owner since the tests
were passing by the fact of matching creator and owner.
Fixes#2445
HTML content was still saved.
This changes makes the back-end check for md content
instead of html to ensure that gets stored in cases
where both are sent to the system.
Closes#2446
- Fixed page editor default focus not working as expected due to
misnamed attribute.
- Added owned_by to relevant areas of the API including the docs.
- Made book relation on page accessible even if deleted since it could cause an issue on views, such as audit trail, when the relation is accessed when the book is deleted.
Found during review of #2353, A revision would be stored before a
restore which would result with a duplicate revision and the new summary
would be assigned against the wrong content.
This change saves the revison after restore and adds test to check the
content and summary text.
Added iframe CSP headers with configuration via .env.
Updated session cookies to be lax by default, dynamically changing to
none when iframes configured to allow third-party control.
Updated cookie security to be auto-secure if a https APP_URL is set.
Related to #2427 and #2207.
Optimised permission fetching so that it won't initialise a bunch
of models for the role permissions and instead does a manual
query to get the data directly.
The content of pages made non-viewable to a user via permissions, within a visible parent, could be seen via the plaintext export option. Before v0.30.6 this would have applied only to scenarios where all pages within the chapter were made non-visible. In v0.30.6 this would make all pages within the chapter visible.
As per #2414
When book trees were generated, pages in chapters where ALL pages within
were not supposed to be visibile, would be visible due to the code
falling back on the raw relation which would not account for
permissions.
This has now been changed so that a custom 'visible_pages' attribute is set and used by any book tree structures, to ensure it does not fall back to the raw relation.
Added an extra test to cover.
For #2414
Tools seems to fit better since the classes were a bit of a mixed bunch
and did not always manage.
Also simplified the structure of the SlugGenerator class.
Also focused EntityContext on shelves and simplified to use session
helper.
Renamed some columns to be more generic and applicable.
Removed now redundant book_id column.
Allowed nullable entity morph columns for non-entity activity.
Ran tests and made required changes.
Repos are generally better since otherwise we end up duplicating
things between front-end and API.
Types moved to by CONST values within a class for better visibilty
of usage and listing of types.
