Skip to content

Commit 0642147

Browse files
jongpiejongpie
authored
Data Masking & String Truncation Improvements (#866)
* Fixed issues in the existing data masking rules for social security numbers, Visa, and Mastercard * Previously, they would incorrectly mask some values that were not the targeted sensitive data * Added new bundled data mask rule for AMEX credit card numbers * Refactored repetitive logic in LogEntryEventBuilder for truncating + masking various text values * Added another round of string truncation after applying data masking to fix issues where string values could still be too long for the corresponding fields * Added several new *Truncated__c fields on LogEntryEvent__e & LogEntry__c for some data points where it would be helpful to have the context that data has been truncated * Added the new *Truncated__c fields on LogEntry__c to the LogEntryRecordPage flexipage * Scope creep: Added more pipeline-only tests for validating the CMDT records bundled with Nebula Logger have the expected values * These tests are intended to validate that the bundled data masking rules work correctly out-of-the-box, but people are free to edit these in their own orgs, so the this level of testing needs to happen in the pipeline only * Scope creep: restructured extra-tests directory to add some organization, things were getting a bit messy * Bumped the managed package's sfdx-project.json file to v4.17.0 (Summer '25 release)
1 parent 323fa77 commit 0642147

254 files changed

Lines changed: 1879 additions & 269 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/build.yml‎

Lines changed: 0 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -174,9 +174,6 @@ jobs:
174174
- name: 'Assign Logger Admin Permission Set'
175175
run: npm run permset:assign:admin
176176

177-
- name: 'Validate Custom Metadata Records'
178-
run: npx sf apex run --file ./scripts/build/validate-custom-metadata-records.apex
179-
180177
# Nebula Logger has functionality that queries the AuthSession object when the current user has an active session.
181178
# The code should work with or without an active session, so the pipeline runs the tests twice - asynchronously and synchronously.
182179
# This is done because, based on how you execute Apex tests, the running user may have an active session (synchrously) or not (asynchronously).
@@ -242,9 +239,6 @@ jobs:
242239
- name: 'Assign Logger Admin Permission Set'
243240
run: npm run permset:assign:admin
244241

245-
- name: 'Validate Custom Metadata Records'
246-
run: npx sf apex run --file ./scripts/build/validate-custom-metadata-records.apex
247-
248242
# Nebula Logger has functionality that queries the AuthSession object when the current user has an active session.
249243
# The code should work with or without an active session, so the pipeline runs the tests twice - asynchronously and synchronously.
250244
# This is done because, based on how you execute Apex tests, the running user may have an active session (synchrously) or not (asynchronously).
@@ -310,9 +304,6 @@ jobs:
310304
- name: 'Assign Logger Admin Permission Set'
311305
run: npm run permset:assign:admin
312306

313-
- name: 'Validate Custom Metadata Records'
314-
run: npx sf apex run --file ./scripts/build/validate-custom-metadata-records.apex
315-
316307
# Nebula Logger has functionality that queries the AuthSession object when the current user has an active session.
317308
# The code should work with or without an active session, so the pipeline runs the tests twice - asynchronously and synchronously.
318309
# This is done because, based on how you execute Apex tests, the running user may have an active session (synchrously) or not (asynchronously).
@@ -381,9 +372,6 @@ jobs:
381372
- name: 'Assign Logger Admin Permission Set'
382373
run: npm run permset:assign:admin
383374

384-
- name: 'Validate Custom Metadata Records'
385-
run: npx sf apex run --file ./scripts/build/validate-custom-metadata-records.apex
386-
387375
# Nebula Logger has functionality that queries the AuthSession object when the current user has an active session.
388376
# The code should work with or without an active session, so the pipeline runs the tests twice - asynchronously and synchronously.
389377
# This is done because, based on how you execute Apex tests, the running user may have an active session (synchrously) or not (asynchronously).
@@ -456,9 +444,6 @@ jobs:
456444
- name: 'Assign Logger Admin Permission Set'
457445
run: npm run permset:assign:admin
458446

459-
- name: 'Validate Custom Metadata Records'
460-
run: npx sf apex run --file ./scripts/build/validate-custom-metadata-records.apex
461-
462447
# Nebula Logger has functionality that queries the AuthSession object when the current user has an active session.
463448
# The code should work with or without an active session, so the pipeline runs the tests twice - asynchronously and synchronously.
464449
# This is done because, based on how you execute Apex tests, the running user may have an active session (synchrously) or not (asynchronously).
@@ -524,9 +509,6 @@ jobs:
524509
- name: 'Assign Logger Admin Permission Set'
525510
run: npm run permset:assign:admin
526511

527-
- name: 'Validate Custom Metadata Records'
528-
run: npx sf apex run --file ./scripts/build/validate-custom-metadata-records.apex
529-
530512
# Nebula Logger has functionality that queries the AuthSession object when the current user has an active session.
531513
# The code should work with or without an active session, so the pipeline runs the tests twice - asynchronously and synchronously.
532514
# This is done because, based on how you execute Apex tests, the running user may have an active session (synchrously) or not (asynchronously).

‎README.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,13 @@
55

66
The most robust observability solution for Salesforce experts. Built 100% natively on the platform, and designed to work seamlessly with Apex, Lightning Components, Flow, OmniStudio, and integrations.
77

8-
## Unlocked Package - v4.16.0
8+
## Unlocked Package - v4.16.1
99

10-
[![Install Unlocked Package in a Sandbox](./images/btn-install-unlocked-package-sandbox.png)](https://test.salesforce.com/packaging/installPackage.apexp?p0=04t5Y0000015pGyQAI)
11-
[![Install Unlocked Package in Production](./images/btn-install-unlocked-package-production.png)](https://login.salesforce.com/packaging/installPackage.apexp?p0=04t5Y0000015pGyQAI)
10+
[![Install Unlocked Package in a Sandbox](./images/btn-install-unlocked-package-sandbox.png)](https://test.salesforce.com/packaging/installPackage.apexp?p0=04tKe0000011MXEIA2)
11+
[![Install Unlocked Package in Production](./images/btn-install-unlocked-package-production.png)](https://login.salesforce.com/packaging/installPackage.apexp?p0=04tKe0000011MXEIA2)
1212
[![View Documentation](./images/btn-view-documentation.png)](https://github.com/jongpie/NebulaLogger/wiki)
1313

14-
`sf package install --wait 20 --security-type AdminsOnly --package 04t5Y0000015pGyQAI`
14+
`sf package install --wait 20 --security-type AdminsOnly --package 04tKe0000011MXEIA2`
1515

1616
---
1717

‎nebula-logger/core.package.xml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,7 @@
156156
<members>LogEntryEvent__e.ExceptionStackTrace__c</members>
157157
<members>LogEntryEvent__e.ExceptionType__c</members>
158158
<members>LogEntryEvent__e.HttpRequestBodyMasked__c</members>
159+
<members>LogEntryEvent__e.HttpRequestBodyTruncated__c</members>
159160
<members>LogEntryEvent__e.HttpRequestBody__c</members>
160161
<members>LogEntryEvent__e.HttpRequestCompressed__c</members>
161162
<members>LogEntryEvent__e.HttpRequestEndpointAddress__c</members>
@@ -164,6 +165,7 @@
164165
<members>LogEntryEvent__e.HttpRequestHeaders__c</members>
165166
<members>LogEntryEvent__e.HttpRequestMethod__c</members>
166167
<members>LogEntryEvent__e.HttpResponseBodyMasked__c</members>
168+
<members>LogEntryEvent__e.HttpResponseBodyTruncated__c</members>
167169
<members>LogEntryEvent__e.HttpResponseBody__c</members>
168170
<members>LogEntryEvent__e.HttpResponseHeaderKeys__c</members>
169171
<members>LogEntryEvent__e.HttpResponseHeaders__c</members>
@@ -248,12 +250,14 @@
248250
<members>LogEntryEvent__e.RecordCollectionType__c</members>
249251
<members>LogEntryEvent__e.RecordId__c</members>
250252
<members>LogEntryEvent__e.RecordJsonMasked__c</members>
253+
<members>LogEntryEvent__e.RecordJsonTruncated__c</members>
251254
<members>LogEntryEvent__e.RecordJson__c</members>
252255
<members>LogEntryEvent__e.RecordSObjectClassification__c</members>
253256
<members>LogEntryEvent__e.RecordSObjectTypeNamespace__c</members>
254257
<members>LogEntryEvent__e.RecordSObjectType__c</members>
255258
<members>LogEntryEvent__e.RequestId__c</members>
256259
<members>LogEntryEvent__e.RestRequestBodyMasked__c</members>
260+
<members>LogEntryEvent__e.RestRequestBodyTruncated__c</members>
257261
<members>LogEntryEvent__e.RestRequestBody__c</members>
258262
<members>LogEntryEvent__e.RestRequestHeaderKeys__c</members>
259263
<members>LogEntryEvent__e.RestRequestHeaders__c</members>
@@ -263,6 +267,7 @@
263267
<members>LogEntryEvent__e.RestRequestResourcePath__c</members>
264268
<members>LogEntryEvent__e.RestRequestUri__c</members>
265269
<members>LogEntryEvent__e.RestResponseBodyMasked__c</members>
270+
<members>LogEntryEvent__e.RestResponseBodyTruncated__c</members>
266271
<members>LogEntryEvent__e.RestResponseBody__c</members>
267272
<members>LogEntryEvent__e.RestResponseHeaderKeys__c</members>
268273
<members>LogEntryEvent__e.RestResponseHeaders__c</members>
@@ -394,6 +399,7 @@
394399
<members>LogEntry__c.HasRestResponseHeaders__c</members>
395400
<members>LogEntry__c.HasStackTrace__c</members>
396401
<members>LogEntry__c.HttpRequestBodyMasked__c</members>
402+
<members>LogEntry__c.HttpRequestBodyTruncated__c</members>
397403
<members>LogEntry__c.HttpRequestBody__c</members>
398404
<members>LogEntry__c.HttpRequestCompressed__c</members>
399405
<members>LogEntry__c.HttpRequestEndpointAddress__c</members>
@@ -402,6 +408,7 @@
402408
<members>LogEntry__c.HttpRequestHeaders__c</members>
403409
<members>LogEntry__c.HttpRequestMethod__c</members>
404410
<members>LogEntry__c.HttpResponseBodyMasked__c</members>
411+
<members>LogEntry__c.HttpResponseBodyTruncated__c</members>
405412
<members>LogEntry__c.HttpResponseBody__c</members>
406413
<members>LogEntry__c.HttpResponseHeaderKeys__c</members>
407414
<members>LogEntry__c.HttpResponseHeaders__c</members>
@@ -490,13 +497,15 @@
490497
<members>LogEntry__c.RecordDetailedLink__c</members>
491498
<members>LogEntry__c.RecordId__c</members>
492499
<members>LogEntry__c.RecordJsonMasked__c</members>
500+
<members>LogEntry__c.RecordJsonTruncated__c</members>
493501
<members>LogEntry__c.RecordJson__c</members>
494502
<members>LogEntry__c.RecordLink__c</members>
495503
<members>LogEntry__c.RecordName__c</members>
496504
<members>LogEntry__c.RecordSObjectClassification__c</members>
497505
<members>LogEntry__c.RecordSObjectTypeNamespace__c</members>
498506
<members>LogEntry__c.RecordSObjectType__c</members>
499507
<members>LogEntry__c.RestRequestBodyMasked__c</members>
508+
<members>LogEntry__c.RestRequestBodyTruncated__c</members>
500509
<members>LogEntry__c.RestRequestBody__c</members>
501510
<members>LogEntry__c.RestRequestHeaderKeys__c</members>
502511
<members>LogEntry__c.RestRequestHeaders__c</members>
@@ -506,6 +515,7 @@
506515
<members>LogEntry__c.RestRequestResourcePath__c</members>
507516
<members>LogEntry__c.RestRequestUri__c</members>
508517
<members>LogEntry__c.RestResponseBodyMasked__c</members>
518+
<members>LogEntry__c.RestResponseBodyTruncated__c</members>
509519
<members>LogEntry__c.RestResponseBody__c</members>
510520
<members>LogEntry__c.RestResponseHeaderKeys__c</members>
511521
<members>LogEntry__c.RestResponseHeaders__c</members>
@@ -700,6 +710,7 @@
700710
<name>CustomIndex</name>
701711
</types>
702712
<types>
713+
<members>LogEntryDataMaskRule.AmericanExpressCreditCardNumber</members>
703714
<members>LogEntryDataMaskRule.MastercardCreditCardNumber</members>
704715
<members>LogEntryDataMaskRule.SocialSecurityNumber</members>
705716
<members>LogEntryDataMaskRule.VisaCreditCardNumber</members>

‎nebula-logger/core/main/configuration/customMetadata/LogEntryDataMaskRule.AmericanExpressCreditCardNumber.md-meta.xml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<CustomMetadata xmlns="http://soap.sforce.com/2006/04/metadata" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
3+
<label>American Express Credit Card Number</label>
4+
<protected>false</protected>
5+
<values>
6+
<field>ApplyToMessage__c</field>
7+
<value xsi:type="xsd:boolean">true</value>
8+
</values>
9+
<values>
10+
<field>ApplyToRecordJson__c</field>
11+
<value xsi:type="xsd:boolean">true</value>
12+
</values>
13+
<values>
14+
<field>IsEnabled__c</field>
15+
<value xsi:type="xsd:boolean">true</value>
16+
</values>
17+
<values>
18+
<field>ReplacementRegEx__c</field>
19+
<value xsi:type="xsd:string">$1****-******-$4</value>
20+
</values>
21+
<values>
22+
<field>SensitiveDataRegEx__c</field>
23+
<value xsi:type="xsd:string">(^|[^0-9A-Za-z])(3[47]\d{2})([- ]?)\d{6}\3(\d{5})(?=[^0-9A-Za-z]|$)</value>
24+
</values>
25+
</CustomMetadata>

‎nebula-logger/core/main/configuration/customMetadata/LogEntryDataMaskRule.MastercardCreditCardNumber.md-meta.xml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,10 @@
1616
</values>
1717
<values>
1818
<field>ReplacementRegEx__c</field>
19-
<value xsi:type="xsd:string">$1****-****-****-$5</value>
19+
<value xsi:type="xsd:string">$1****-****-****-$4</value>
2020
</values>
2121
<values>
2222
<field>SensitiveDataRegEx__c</field>
23-
<value xsi:type="xsd:string">(^|[ ])(5\d{3})[- ]*(\d{4})[- ]*(\d{4})[- ]*(\d{4})</value>
23+
<value xsi:type="xsd:string">(^|[^0-9])(5[1-5]\d{2}|222[1-9]|22[3-9]\d|2[3-6]\d{2}|27[01]\d|2720)([- ]?)\d{4}\3\d{4}\3(\d{4})(?!\d)</value>
2424
</values>
2525
</CustomMetadata>

‎nebula-logger/core/main/configuration/customMetadata/LogEntryDataMaskRule.SocialSecurityNumber.md-meta.xml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,10 @@
1616
</values>
1717
<values>
1818
<field>ReplacementRegEx__c</field>
19-
<value xsi:type="xsd:string">$1XXX-XX-$4$5</value>
19+
<value xsi:type="xsd:string">$1XXX-XX-$4</value>
2020
</values>
2121
<values>
2222
<field>SensitiveDataRegEx__c</field>
23-
<value xsi:type="xsd:string">(^|[ ])(\d{3})[- ]*(\d{2})[- ]*(\d{4})([ ]|$)</value>
23+
<value xsi:type="xsd:string">(^|[^0-9A-Za-z])(\d{3})[- ]?(\d{2})[- ]?(\d{4})(?=[^0-9A-Za-z]|$)</value>
2424
</values>
2525
</CustomMetadata>

‎nebula-logger/core/main/configuration/customMetadata/LogEntryDataMaskRule.VisaCreditCardNumber.md-meta.xml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,10 @@
1616
</values>
1717
<values>
1818
<field>ReplacementRegEx__c</field>
19-
<value xsi:type="xsd:string">$1****-****-****-$5</value>
19+
<value xsi:type="xsd:string">$1****-****-****-$4</value>
2020
</values>
2121
<values>
2222
<field>SensitiveDataRegEx__c</field>
23-
<value xsi:type="xsd:string">(^|[ ])(4\d{3})[- ]*(\d{4})[- ]*(\d{4})[- ]*(\d{4})</value>
23+
<value xsi:type="xsd:string">(^|[^0-9])(4\d{3})([- ]?)\d{4}\3\d{4}\3(\d{4})(?!\d)</value>
2424
</values>
2525
</CustomMetadata>

‎nebula-logger/core/main/log-management/classes/LogEntryEventHandler.cls‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -287,6 +287,7 @@ public without sharing class LogEntryEventHandler extends LoggerSObjectHandler {
287287
ExceptionType__c = logEntryEvent.ExceptionType__c,
288288
HttpRequestBody__c = logEntryEvent.HttpRequestBody__c,
289289
HttpRequestBodyMasked__c = logEntryEvent.HttpRequestBodyMasked__c,
290+
HttpRequestBodyTruncated__c = logEntryEvent.HttpRequestBodyTruncated__c,
290291
HttpRequestCompressed__c = logEntryEvent.HttpRequestCompressed__c,
291292
HttpRequestEndpoint__c = logEntryEvent.HttpRequestEndpoint__c,
292293
HttpRequestEndpointAddress__c = logEntryEvent.HttpRequestEndpointAddress__c,
@@ -295,6 +296,7 @@ public without sharing class LogEntryEventHandler extends LoggerSObjectHandler {
295296
HttpRequestMethod__c = logEntryEvent.HttpRequestMethod__c,
296297
HttpResponseBody__c = logEntryEvent.HttpResponseBody__c,
297298
HttpResponseBodyMasked__c = logEntryEvent.HttpResponseBodyMasked__c,
299+
HttpResponseBodyTruncated__c = logEntryEvent.HttpResponseBodyTruncated__c,
298300
HttpResponseHeaderKeys__c = logEntryEvent.HttpResponseHeaderKeys__c,
299301
HttpResponseHeaders__c = logEntryEvent.HttpResponseHeaders__c,
300302
HttpResponseStatus__c = logEntryEvent.HttpResponseStatus__c,
@@ -349,11 +351,13 @@ public without sharing class LogEntryEventHandler extends LoggerSObjectHandler {
349351
RecordId__c = logEntryEvent.RecordId__c,
350352
RecordJson__c = logEntryEvent.RecordJson__c,
351353
RecordJsonMasked__c = logEntryEvent.RecordJsonMasked__c,
354+
RecordJsonTruncated__c = logEntryEvent.RecordJsonTruncated__c,
352355
RecordSObjectClassification__c = logEntryEvent.RecordSObjectClassification__c,
353356
RecordSObjectType__c = logEntryEvent.RecordSObjectType__c,
354357
RecordSObjectTypeNamespace__c = logEntryEvent.RecordSObjectTypeNamespace__c,
355358
RestRequestBody__c = logEntryEvent.RestRequestBody__c,
356359
RestRequestBodyMasked__c = logEntryEvent.RestRequestBodyMasked__c,
360+
RestRequestBodyTruncated__c = logEntryEvent.RestRequestBodyTruncated__c,
357361
RestRequestHeaderKeys__c = logEntryEvent.RestRequestHeaderKeys__c,
358362
RestRequestHeaders__c = logEntryEvent.RestRequestHeaders__c,
359363
RestRequestMethod__c = logEntryEvent.RestRequestMethod__c,
@@ -363,6 +367,7 @@ public without sharing class LogEntryEventHandler extends LoggerSObjectHandler {
363367
RestRequestUri__c = logEntryEvent.RestRequestUri__c,
364368
RestResponseBody__c = logEntryEvent.RestResponseBody__c,
365369
RestResponseBodyMasked__c = logEntryEvent.RestResponseBodyMasked__c,
370+
RestResponseBodyTruncated__c = logEntryEvent.RestResponseBodyTruncated__c,
366371
RestResponseHeaderKeys__c = logEntryEvent.RestResponseHeaderKeys__c,
367372
RestResponseHeaders__c = logEntryEvent.RestResponseHeaders__c,
368373
RestResponseStatusCode__c = logEntryEvent.RestResponseStatusCode__c,

0 commit comments

Comments
 (0)