Refactor parseArtSci function and add tests for newly introduced parseDepartmentList

[r-weng]

Proposed Changes

This pull request refactors the parseArtSci function in app/WebParsing/ArtSciParser.hs by introducing a new function, parseDepartmentList, to handle parsing of department lists. parseDepartmentList now ignores department pages with no courses, duplicate courses, and program areas belonging to a college. Additionally, this pull request adds test for the newly introduced parseDepartmentList function.

Type of Change

(Write an X or a brief description next to the type or types that best describe your changes.)

Type	Applies?
🚨 Breaking change (fix or feature that would cause existing functionality to change)
✨ New feature (non-breaking change that adds functionality)
🐛 Bug fix (non-breaking change that fixes an issue)
🎨 User interface change (change to user interface; provide screenshots)
♻️ Refactoring (internal change to codebase, without changing functionality)	X
🚦 Test update (change that only adds or modifies tests)
📦 Dependency update (change that updates a dependency)
🔧 Internal (change that only affects developers or continuous integration)

Checklist

(Complete each of the following items for your pull request. Indicate that you have completed an item by changing the [ ] into a [x] in the raw text, or by clicking on the checkbox in the rendered description on GitHub.)

Before opening your pull request:

• I have performed a self-review of my changes.
  • Check that all changed files included in this pull request are intentional changes.
  • Check that all changes are relevant to the purpose of this pull request, as described above.
• I have added tests for my changes, if applicable.
  • This is required for all bug fixes and new features.
• I have updated the project documentation, if applicable.
  • This is required for new features.
• If this is my first contribution, I have added myself to the list of contributors.
• I have updated the project Changelog (this is required for all changes).

After opening your pull request:

• I have verified that the CircleCI checks have passed.
• I have requested a review from a project maintainer.

Questions and Comments

(Include any questions or comments you have regarding your changes.)

[coveralls]

Coverage Report for CI Build 696

Coverage increased (+0.4%) to 58.964%

Details

• Coverage increased (+0.4%) from the base build.
• Patch coverage: 1 uncovered change across 1 file (12 of 13 lines covered, 92.31%).
• No coverage regressions found.

Uncovered Changes

File	Changed	Covered	%
app/WebParsing/ArtSciParser.hs	13	12	92.31%

Coverage Regressions

No coverage regressions found.

---

Coverage Stats

Relevant Lines:	3961
Covered Lines:	2401
Line Coverage:	60.62%
Relevant Branches:	964
Covered Branches:	503
Branch Coverage:	52.18%
Branches in Coverage %:	Yes
Coverage Strength:	164.24 hits per line

---

💛 - Coveralls

[david-yz-liu]

@r-weng This is good, but please write tests for this new function

[david-yz-liu]

It isn't necessary to add a second entry describing test cases added (this is implied as part of the first change)

[david-yz-liu]

First, move this into getDeptList (i.e., we don't need to separate out the "cleaning" step). That function currently only calls T.strip, so it can be simplified by using getDeptList anyways.

Second, rather than do this replacing, please use the existing cleanText. That said, replacing "\160" with " " is better than replacing it with "", so you can also go ahead and modify cleanText itself. (See here for a reference for the '\160' char.)

[david-yz-liu]

As was mentioned at our meeting yesterday, this filtering is a great candidate for a local helper function. The idiomatic way to define this is to use a where block in this function:

    return filterDepartments deptList
    where
        filterDepartments :: [(T.Text, T.Text)] -> [(T.Text, T.Text)]
        ...

[david-yz-liu]

While parseDepartmentList is good to be exported for testing purposes, getDeptList isn't used anywhere else in the codebase so we can remove it from this export list.