Skip to content

Commit 1b3ed1b

Browse files
adliusantkrytihorsokhanexoftJohnetordofffuta-ikeda
authored
Feature/pbs 25 13 (#11253)
* handle django core validation error in drf view * add unit tests * add registration_word for registration provider * [ENG-8290] Allow collection search POST with token scope (#11201) Purpose Add scope for POST collection search * [ENG-8247] Ability to delete draft preprints from database (#11191) ## Purpose User should be able to delete draft preprints, so that we don't save them in database ## Changes Added `delete` method. User can delete a preprint only if it's in initial state, so it means this preprint is a draft ## Ticket https://openscience.atlassian.net/browse/ENG-8247 * [ENG-8193] Fix issues with Preprint submission via API (#11185) ## Purpose handle django core ValidationError in drf view ## Ticket https://openscience.atlassian.net/browse/ENG-8193 * [ENG-8192] Ability to force archive registrations when OSFS Folders have changed (#11194) ## Purpose fix force archive for some actions ## Changes - divide revert_log_actions into separate functions - include trashed children when build a file tree - add additional retrieval logic for osf_storage_folder_created and osf_storage_file_removed - add generic retrieval fallback ## Ticket https://openscience.atlassian.net/browse/ENG-8192 * add additional information to user admin (#11184) ## Purpose add more information to user admin ## Changes Added fields: - id - Deactivation request - Change password last attempt - Change password invalid attempts - Socials - User is staff - Groups Added dates to: - Disabled - Registered - Confirmed ## Ticket https://openscience.atlassian.net/browse/ENG-8190 * upgrade django to 4.2.17 (#11173) ## Purpose Because Django 4.2.15 version has a vulnerability, it was upgraded to 4.2.17 ## Changes Updated pyproject.toml and lock file ## Ticket https://openscience.atlassian.net/browse/ENG-8176 * added retry to avoid race condition (#11179) ## Purpose User received an email of archive failure regardless of it was successful. It may happen because `archive_success` is run asynchronously https://github.com/CenterForOpenScience/osf.io/blob/2328dd60f55e9c1281dcb29dcd45a78a7fd2cc5f/website/archiver/listeners.py#L33-L49 and may be finished before the main thread finishes archiving process. So at first `archive_success` is processed and no files found, thus email is sent, then the main thread finishes files processing and this archiving is successful actually. Also it's possible that the celery queue had too many tasks to process and when the main thread finishes archiving, user sees his registration and when celery processes `archive_success` tasks that fails, user receives this email. ## Changes Added a one-time retry. ## Ticket https://openscience.atlassian.net/browse/ENG-8175?atlOrigin=eyJpIjoiMjg4MWM1YWI1ZTE3NDMyZmEyODk2Y2QxZjlhNjFlOGQiLCJwIjoiaiJ9 * [ENG-8096] Admins on projects are unable to reject user access requests (#11163) ## Purpose Add default value when reject access request ## Ticket https://openscience.atlassian.net/browse/ENG-8096 * fix content overflow for node page (#11182) ## Purpose fix content overflow on the node admin page ## Ticket https://openscience.atlassian.net/browse/ENG-8063 * don't add multiple group perms for preprint provider (#11159) ## Purpose don't add multiple group perms for preprint provider ## Changes - check if user already belongs to some provider group - change checkbox to radio select ## Ticket https://openscience.atlassian.net/browse/ENG-8016 * fixed children/parent fields in admin templates (#11156) ## Purpose Admin templates used nonexistent fields to display children and parent of a node (potentially new fields were added but the old fields weren't replaced by new ones). The templates used `parent` and `children` fields. However an endpoint that adds children and parent to a node uses fields `descendants` (or through `NodeRelation` using `get_nodes` method) and `parent_node` property through `_parents` field ## Changes Use correct fields ## Notes Deletion of children nodes that are displayed now is broken. Together with Mark decided to create a separate ticket for this issue ## Ticket https://openscience.atlassian.net/browse/ENG-7969 * [ENG-7962] Fix User Setting Response Payload async mailchimp perference change issues (#11136) ## Purpose Correct issues where PATCH responses wouldn't show preferences as updated due to async mailchimp task execution. Overrides instance model to return changed value, even though db value is unchanged until mailchimp confirms. ## Changes - adds special case for returning desired value synchronously - removes `update_osf_help_mails_subscription` because it's not used, but tested. - adds test cases ## Ticket https://openscience.atlassian.net/browse/ENG-7962 * improved displaying of stashed urls and approval state in admin (#11193) ## Purpose Admins feel discomfort while looking for tokens in stashed urls and approval state fields on a registration page in admin ## Changes Format fields in a json-like way ## Ticket https://openscience.atlassian.net/browse/ENG-6614?atlOrigin=eyJpIjoiMTdkMTM0YzBmNjljNDcxYTkxNjYzZDRjYTY3NjEyODMiLCJwIjoiaiJ9 * [ENG-5862] SPAM - Fix Wiki Spamming (#11171) <!-- Before submit your Pull Request, make sure you picked the right branch: - For hotfixes, select "master" as the target branch - For new features, select "develop" as the target branch - For release feature fixes, select the relevant release branch (release/X.Y.Z) as the target branch --> ## Purpose verify if spammy domains are detected ## Changes - merge check_resource_for_domains_postcommit and check_resource_with_spam_services tasks to avoid race condition - compare note to value not enum - log detected domains to sentry ## QA Notes You can test it with domain [xakw1.com](https://admin.staging3.osf.io/admin/osf/notabledomain/1180/change/?_changelist_filters=q%3Dxakw1.com) on staging3. Currently project won't be banned with this domain, regardless of whether it's public or not. ## Ticket https://openscience.atlassian.net/browse/ENG-5862 * switch to new UI when user views draft registration file (#11144) ## Purpose Throughout registration creation user can view attached files. However `resolve_guid` doesn't handle this case and renders the legacy UI that shows error because `get_rdf_type` raises `NotImplementedError`. So this case has no be handled by ember. However when we switch to the new UI, we get `draftnode is not a supported target type`. Also there will be some work for FE because for now ember relies on `node` relationship that draft node/registration don't have. Taking into account Futa's words, it's an unusual flow for ember ## Changes Added redirect to ember, updated `view_map` that referenced to the non-existing view `draft_nodes:node-detail` to use `draft_nodes:draft-node-detail` view ## Notes 1. I'm not confident that `draft-node` key is still used in `view_map`. From the `resolve` method of `TargetField` I see that keys are either model name in lowercase or `referent._name` that I couldn't find how is created (maybe automatically). However if it existed, there would be some errors that this view does not exist. So I left both keys but with the correct view name. In case you know the answer, I can remove the original key and leave only the correct one ## Ticket https://openscience.atlassian.net/jira/software/c/projects/ENG/boards/145?selectedIssue=ENG-5810 * [ENG-7929] Ability to move registrations to draft state (#11153) ## Purpose Admins should be able to revert registration to draft state for the support purposes ## Changes Added functionality to revert registration back to draft state Added a button in admin to revert a registration Added tests ## Notes: Currently we are discussing with Mark what to do with a registration after admin reverts it, so this ticket cannot be merged. The issue is that when admin reverts registration, it's still displayed for admin, just with deletion date and when user re-registers a draft, the system creates a new registration and the draft is linked to this newest version and admin sees both registrations. So if admin tries to revert the previous version, he'll get an error because the version doesn't have the linked draft. What I've suggested: 1. Fully delete previous registered version so that you won't be able to see it in admin. In this way if the registered version had guid 123ab and user registered the restored draft again, the draft would have another registration with another guid, like 999ss . Basically for user it'll be the same registration, just with another url to access it 2. Add another (or the same) hint for Revert to Draft button that you can't revert this previous version (the easiest solution) 3. One more solution, that I think will be the most complicated between these three, is that we can try to save the guid of an original registered version and whenever admin reverts registration and user registers it again from the same draft, we assign the saved guid to a new registration **DECISION**: Option 2 is the most reliable. In case we want #1 or #3 behavior, a new ticket will be created ## Ticket https://openscience.atlassian.net/browse/ENG-7929 * added a route to download node metadata (#11215) ## Purpose So right now, our firewall rules aren't advanced enough to be able to let requests from `/<guid>/metadata` go through to osf on staging4. Instead, we have to start with a stable string, and do the dynamic part afterward, hence doing `/metadata/<guid>/` which we can match with `/metadata/*` in the firewall. ## Changes Added a corresponding route ## Ticket https://openscience.atlassian.net/browse/ENG-8261 * add exception handling to /review_actions/ endpoint * [ENG-8246] Fixed deletion of maintenance alerts in admin (#11226) What When you try to delete the active maintenance banner from the admin app, it 502s. The traceback says that there is no url to redirect to and to provide a success url. Acceptance Criteria A user will be able to add a maintenance banner and then delete it. When the banner is added, it will show on the OSF dashboard page (among other places). When deleted, it will no longer show. * [ENG-8325] Public column does not display the visibility status of child nodes on the Nodes page in the Admin App Public column does not display the visibility status of child nodes on the Nodes page in the Admin App * API: Allow /v2/users/me/preprints list view to filter by title Allow the User Preprints endpoint (/v2/users/me/preprints) to be filterable by title (?filter[title]=example), similar to how the User Nodes endpoint (/v2/users/me/nodes) supports this filter capability * [ENG-8224] Fixed force archive template with registration addons (#11210) Enable Product Team to Force Archive Registrations in the Admin App * add brand relationship to collectionprovider * Add collections scopes to FULL_READ and FULL_WRITE * [ENG-8936] API: Allow /v2/users/me/preprints list view to filter by tags (#11232) * fix categories for sendgrid emails (#11236) * [ENG-8401] Fixed preprint downloading (#11238) * fixed preprint downloading * fixed nonetype * [ENG-8216] Fixed children deletion on a node page in admin (#11237) * field children deletion in admin * improved performance and removed unused attributes * [ENG-7979] Registrations pending moderation that have components also pending moderation do not display the children (#11222) * show only public nodes for non-authorized users in the node queryset * add custom filters to can_view(); include pending nodes for moderators in get_node_count() * add test * added academiaInstitution in social-schema, fixed True value of 'ongoing', fixed/added tests (#11239) ## Purpose V2 API doesn't allow setting `True` value for `ongoing` property in employment/education tabs and set `academiaInstitution` property in social tab ## Changes Added `academiaInstitution` field in social-schema Fixed ignored required properties for `ongoing` property in employment/education-schema files Added new tests and fixed the old ones ## QA Notes Can be tested only via API Updates can be viewed in user settings ## Ticket https://openscience.atlassian.net/browse/ENG-8455 * [ENG-8401] Earlier preprint versions download the current file (#11245) * fixed earlier version download the newest version file * fixed tests * [ENG-8462] Institution setup fixes (#11241) * create monthly reports when institution is created * better exception handling; async generate report * fix test * handle deactivated institutions; minor fixes * [Reopen] [ENG-8192] Ability to force archive registrations when OSF Folders have changed (#11247) * break force_archive into logical parts; support trashed children * add unit tests; minor fixes * fix name collisions issue and general fallback * add test * minor fixes --------- Co-authored-by: Anton Krytskyi <[email protected]> Co-authored-by: ihorsokhanexoft <[email protected]> Co-authored-by: John Tordoff <[email protected]> Co-authored-by: futa-ikeda <[email protected]>
1 parent f7df09c commit 1b3ed1b

File tree

78 files changed

+1767
-482
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

78 files changed

+1767
-482
lines changed

addons/base/views.py

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1006,13 +1006,17 @@ def persistent_file_download(auth, **kwargs):
10061006
file = BaseFileNode.active.filter(_id=id_or_guid).first()
10071007
if not file:
10081008
guid = Guid.load(id_or_guid)
1009-
if guid:
1010-
file = guid.referent
1011-
else:
1009+
if not guid:
10121010
raise HTTPError(http_status.HTTP_404_NOT_FOUND, data={
10131011
'message_short': 'File Not Found',
10141012
'message_long': 'The requested file could not be found.'
10151013
})
1014+
1015+
file = guid.referent
1016+
if type(file) is Preprint:
1017+
referent, _ = Guid.load_referent(id_or_guid)
1018+
file = referent.primary_file
1019+
10161020
if not file.is_file:
10171021
raise HTTPError(http_status.HTTP_400_BAD_REQUEST, data={
10181022
'message_long': 'Downloading folders is not permitted.'

admin/institutions/urls.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
re_path(r'^import/$', views.ImportInstitution.as_view(), name='import'),
1010
re_path(r'^(?P<institution_id>[0-9]+)/$', views.InstitutionDetail.as_view(), name='detail'),
1111
re_path(r'^(?P<institution_id>[0-9]+)/export/$', views.InstitutionExport.as_view(), name='export'),
12+
re_path(r'^(?P<institution_id>[0-9]+)/monthly_report/$', views.InstitutionMonthlyReporterDo.as_view(), name='monthly_report'),
1213
re_path(r'^(?P<institution_id>[0-9]+)/delete/$', views.DeleteInstitution.as_view(), name='delete'),
1314
re_path(r'^(?P<institution_id>[0-9]+)/deactivate/$', views.DeactivateInstitution.as_view(), name='deactivate'),
1415
re_path(r'^(?P<institution_id>[0-9]+)/reactivate/$', views.ReactivateInstitution.as_view(), name='reactivate'),

admin/institutions/views.py

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import json
2+
from dateutil.parser import isoparse
23

34
from django.contrib import messages
45
from django.contrib.auth.mixins import PermissionRequiredMixin
@@ -15,6 +16,9 @@
1516
from admin.base.forms import ImportFileForm
1617
from admin.institutions.forms import InstitutionForm, InstitutionalMetricsAdminRegisterForm
1718
from osf.models import Institution, Node, OSFUser
19+
from osf.metrics.utils import YearMonth
20+
from osf.metrics.reporters import AllMonthlyReporters
21+
from osf.management.commands.monthly_reporters_go import monthly_reporter_do
1822

1923

2024
class InstitutionList(PermissionRequiredMixin, ListView):
@@ -129,6 +133,38 @@ def get(self, request, *args, **kwargs):
129133
return response
130134

131135

136+
class InstitutionMonthlyReporterDo(PermissionRequiredMixin, View):
137+
permission_required = 'osf.view_institution'
138+
raise_exception = True
139+
140+
def post(self, request, *args, **kwargs):
141+
institution_id = self.kwargs.get('institution_id')
142+
try:
143+
institution = Institution.objects.get_all_institutions().get(id=institution_id)
144+
except Institution.DoesNotExist:
145+
raise Http404(f"Institution with id {institution_id} is not found or deactivated.")
146+
147+
monthly_report_date = request.POST.get('monthly_report_date', None)
148+
if monthly_report_date:
149+
try:
150+
monthly_report_date = isoparse(monthly_report_date).date()
151+
except ValueError as exc:
152+
messages.error(request, str(exc))
153+
return redirect('institutions:detail', institution_id=institution.id)
154+
else:
155+
messages.error(request, 'Report date cannot be none.')
156+
return redirect('institutions:detail', institution_id=institution.id)
157+
158+
monthly_reporter_do.apply_async(kwargs={
159+
'yearmonth': str(YearMonth.from_date(monthly_report_date)),
160+
'reporter_key': request.POST.get('monthly_reporter', None),
161+
'report_kwargs': {'institution_pk': institution.id},
162+
})
163+
164+
messages.success(request, 'Monthly reporter successfully went.')
165+
return redirect('institutions:detail', institution_id=institution.id)
166+
167+
132168
class CreateInstitution(PermissionRequiredMixin, CreateView):
133169
permission_required = 'osf.change_institution'
134170
raise_exception = True
@@ -141,6 +177,18 @@ def get_context_data(self, *args, **kwargs):
141177
kwargs['import_form'] = ImportFileForm()
142178
return super().get_context_data(*args, **kwargs)
143179

180+
def form_valid(self, form):
181+
response = super().form_valid(form)
182+
183+
# Make a report after Institution is created
184+
monthly_reporter_do.apply_async(kwargs={
185+
'yearmonth': str(YearMonth.from_date(self.object.created)),
186+
'reporter_key': AllMonthlyReporters.INSTITUTIONAL_SUMMARY.name,
187+
'report_kwargs': {'institution_pk': self.object.id},
188+
})
189+
190+
return response
191+
144192

145193
class InstitutionNodeList(PermissionRequiredMixin, ListView):
146194
template_name = 'institutions/node_list.html'

admin/maintenance/views.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
from admin.maintenance.forms import MaintenanceForm
77

88
from django.shortcuts import redirect
9+
from django.urls import reverse_lazy
910
from django.forms.models import model_to_dict
1011
from django.views.generic import DeleteView, TemplateView
1112
from django.contrib.auth.mixins import PermissionRequiredMixin
@@ -15,11 +16,13 @@ class DeleteMaintenance(PermissionRequiredMixin, DeleteView):
1516
permission_required = 'osf.delete_maintenancestate'
1617
raise_exception = True
1718
template_name = 'maintenance/delete_maintenance.html'
19+
success_url = reverse_lazy('maintenance:display')
1820

1921
def get_object(self, queryset=None):
2022
return MaintenanceState.objects.first()
2123

22-
def delete(self, request, *args, **kwargs):
24+
def post(self, request, *args, **kwargs):
25+
super().post(request, *args, **kwargs)
2326
maintenance.unset_maintenance()
2427
return redirect('maintenance:display')
2528

admin/management/views.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,7 @@ def post(self, request, *args, **kwargs):
130130
if report_date is not None
131131
else ''
132132
),
133+
reporter_key=request.POST.get('monthly_reporter', '')
133134
)
134135

135136
if errors:

admin/nodes/urls.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,4 +45,5 @@
4545
re_path(r'^(?P<guid>[a-z0-9]+)/remove_notifications/$', views.NodeRemoveNotificationView.as_view(), name='node-remove-notifications'),
4646
re_path(r'^(?P<guid>[a-z0-9]+)/update_moderation_state/$', views.NodeUpdateModerationStateView.as_view(), name='node-update-mod-state'),
4747
re_path(r'^(?P<guid>[a-z0-9]+)/resync_datacite/$', views.NodeResyncDataCiteView.as_view(), name='resync-datacite'),
48+
re_path(r'^(?P<guid>[a-z0-9]+)/revert/$', views.NodeRevertToDraft.as_view(), name='revert-to-draft'),
4849
]

admin/nodes/views.py

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@
1414
View,
1515
FormView,
1616
ListView,
17-
TemplateView,
1817
)
1918
from django.shortcuts import redirect, reverse, get_object_or_404
2019
from django.urls import reverse_lazy
@@ -102,11 +101,16 @@ def get_context_data(self, **kwargs):
102101
node = self.get_object()
103102

104103
detailed_duplicates = detect_duplicate_notifications(node_id=node.id)
105-
104+
children = node.get_nodes(is_node_link=False)
105+
# Annotate guid because django templates prohibit accessing attributes that start with underscores
106+
children = AbstractNode.objects.filter(
107+
id__in=[child.id for child in children]
108+
).prefetch_related('guids').annotate(guid=F('guids___id'))
106109
context.update({
107110
'SPAM_STATUS': SpamStatus,
108111
'STORAGE_LIMITS': settings.StorageLimits,
109112
'node': node,
113+
'children': children,
110114
'duplicates': detailed_duplicates
111115
})
112116

@@ -193,10 +197,9 @@ def add_contributor_removed_log(self, node, user):
193197
).save()
194198

195199

196-
class NodeDeleteView(NodeMixin, TemplateView):
200+
class NodeDeleteView(NodeMixin, View):
197201
""" Allows authorized users to mark nodes as deleted.
198202
"""
199-
template_name = 'nodes/remove_node.html'
200203
permission_required = ('osf.view_node', 'osf.delete_node')
201204
raise_exception = True
202205

@@ -761,7 +764,7 @@ def post(self, request, *args, **kwargs):
761764

762765
allow_unconfigured = force_archive_params.get('allow_unconfigured', False)
763766

764-
addons = set(force_archive_params.getlist('addons', []))
767+
addons = set(registration.registered_from.get_addon_names())
765768
addons.update(DEFAULT_PERMISSIBLE_ADDONS)
766769

767770
try:
@@ -780,8 +783,8 @@ def post(self, request, *args, **kwargs):
780783
registration,
781784
permissible_addons=addons,
782785
allow_unconfigured=allow_unconfigured,
783-
skip_collision=skip_collision,
784-
delete_collision=delete_collision,
786+
skip_collisions=skip_collision,
787+
delete_collisions=delete_collision,
785788
)
786789
messages.success(request, 'Registration archive process has finished.')
787790
except Exception as exc:
@@ -800,3 +803,12 @@ def post(self, request, *args, **kwargs):
800803
registration = self.get_object()
801804
registration.request_identifier_update('doi', create=True)
802805
return redirect(self.get_success_url())
806+
807+
808+
class NodeRevertToDraft(NodeMixin, View):
809+
permission_required = 'osf.change_node'
810+
811+
def post(self, request, *args, **kwargs):
812+
registration = self.get_object()
813+
registration.to_draft()
814+
return redirect(self.get_success_url())

admin/preprint_providers/forms.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -116,10 +116,10 @@ def __init__(self, *args, provider_groups=None, **kwargs):
116116
super().__init__(*args, **kwargs)
117117

118118
provider_groups = provider_groups or Group.objects.none()
119-
self.fields['group_perms'] = forms.ModelMultipleChoiceField(
119+
self.fields['group_perms'] = forms.ModelChoiceField(
120120
queryset=provider_groups,
121121
required=False,
122-
widget=forms.CheckboxSelectMultiple
122+
widget=forms.RadioSelect
123123
)
124124

125125
user_id = forms.CharField(required=True, max_length=5, min_length=5)

admin/preprint_providers/views.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -481,10 +481,14 @@ def form_valid(self, form):
481481
if not osf_user:
482482
raise Http404(f'OSF user with id "{user_id}" not found. Please double check.')
483483

484-
for group in form.cleaned_data.get('group_perms'):
485-
self.target_provider.add_to_group(osf_user, group)
484+
if osf_user.has_groups(self.target_provider.group_names):
485+
messages.error(self.request, f'User with guid: {user_id} is already a moderator or admin')
486+
return super().form_invalid(form)
486487

488+
group = form.cleaned_data.get('group_perms')
489+
self.target_provider.add_to_group(osf_user, group)
487490
osf_user.save()
491+
488492
messages.success(self.request, f'Permissions update successful for OSF User {osf_user.username}!')
489493
return super().form_valid(form)
490494

admin/providers/views.py

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,7 @@ def post(self, request, *args, **kwargs):
2929
messages.error(request, f'User for guid: {data["add-moderators-form"][0]} could not be found')
3030
return redirect(f'{self.url_namespace}:add_admin_or_moderator', provider_id=provider.id)
3131

32-
groups = [provider.format_group(name) for name in provider.groups.keys()]
33-
if target_user.has_groups(groups):
32+
if target_user.has_groups(provider.group_names):
3433
messages.error(request, f'User with guid: {data["add-moderators-form"][0]} is already a moderator or admin')
3534
return redirect(f'{self.url_namespace}:add_admin_or_moderator', provider_id=provider.id)
3635

0 commit comments

Comments
 (0)