mirror of
https://github.com/hanzoai/laya.git
synced 2026-10-30 19:33:31 +00:00
The English quote-header marker was `^\s*From:\s.+$`, which matches any line that starts `From: `. Ordinary prose opens that way, so the request was deleted and the model answered an empty body — the same silent failure this module has been fixed for twice already (#94, #227): ```python from laya import email body = ("Hi team,\n" "From: my side the integration works, but please refund the duplicate charge today.\n" "Thanks") print(email.clean_email_body(body)) # before # 'Hi team,' # -> "refund" in cleaned is False; the model never sees the request # after # 'Hi team,\nFrom: my side the integration works, but please refund the duplicate charge today.\nThanks' ``` The `De:` marker two lines below already had this exact fix, for the same reason and with a comment saying so — `De:` also opens ordinary Portuguese/Spanish lines. `From:` opens ordinary English lines, and a reply header always carries the sender, so it is now gated on an address the same way: ```python re.compile(r"^\s*From:\s.*[@<]", re.I), # was r"^\s*From:\s.+$" ``` Deliberate choices tested: - **Gate on an address rather than on the shape of the text.** A word-count or capitalisation rule would be guessing at what a sentence looks like; "a reply header names its sender" is a property of the header, and it is the rule `De:` already uses, so the two now read the same way instead of one being stricter for no stated reason. - **A bare `From: Name` header is still cut, by its neighbours.** This is the part I got wrong first: I assumed `_SIGNATURE_MARKERS` covered `Sent:`, but `_HEADER_NEXT` only listed the Portuguese/Spanish `Enviado:`/`Data:`/`Fecha:` forms, so dropping the address from the marker leaked every English quote header written without one. `_HEADER_FROM_NAME`/`_HEADER_NEXT` now accept the English pair too — `From: <name>` followed by `Sent:` or a dated `Date:` — which is the translation of the rule that was already there for `De:`. - **`To:` alone is still not enough**, matching the existing note that "`De: 10/09 / Para: 15/09` is how a leave request reads". `From: Alice Smith` followed by `To: Bob` and a request is kept, because a `To:` line is not evidence of a quote block. - **When it cannot be told apart, the request wins.** A bare `From: Name` with no header neighbour is kept. A leaked quote is noise the model can read past; a deleted request is a wrong answer, which is the asymmetry #94 and #227 were both about. The suite had **no `From:` case at all** before this, which is why it stayed green on the bug — `python tests/test_email.py` reported `90 passed, 0 failed` on `main` with the bug present. The new block covers five prose forms, the four quoted-header forms that must still cut, and the two ambiguous forms that must be kept: ``` # with the marker reverted to r"^\s*From:\s.+$" 98 passed, 7 failed FAIL en/`From:` prose keeps the request: the reported case Hi team, FAIL en/`From:` prose keeps the request: lowercase Hi, FAIL en/`From:` prose keeps the request: uppercase Hi, FAIL en/`From:` prose keeps the request: extra spacing Hi, FAIL en/`From:` prose keeps the request: mid-body Hello, FAIL en/bare `From:` with no header neighbour is kept: From: Name then To: Hi, FAIL en/bare `From:` with no header neighbour is kept: From: Name alone Hi, # restored 105 passed, 0 failed ``` The four quoted-header positive controls stay green in both states, so the fix is not simply "stop cutting on `From:`". ``` $ python tests/test_email.py 105 passed, 0 failed (90 before) $ python tests/test_lang_guess.py 52 passed, 0 failed $ python tests/test_router.py all routing tests passed $ python tests/test_criteria.py 128 passed, 0 failed $ python tests/test_packaging.py all packaging tests passed $ python -m ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120 All checks passed! $ python -m compileall -q laya/ tests/ ``` Known limitation: `From:` followed by an address but no other header line is treated as a quote header and cut, because that is what an address on a `From:` line overwhelmingly is in an email body. A caller who writes prose that literally begins `From: <[email protected]>` loses that line; I could not construct a realistic case where that is a request, and the quoted-header case is the common one.
508 lines
22 KiB
Python
508 lines
22 KiB
Python
"""Email cleaning: a disclaimer footer must not delete the sender's actual request.
|
|
|
|
Regression tests for `laya.email.clean_email_body`. `_DISCLAIMER` used to be applied to whole
|
|
paragraphs, so any paragraph that merely *mentioned* boilerplate was deleted outright. When the
|
|
footer ran on without a blank line, the request went with it:
|
|
|
|
clean_email_body("My account is locked.\\nThis email is confidential...\\nPlease unlock it.")
|
|
# before: '' <- the whole body, request included, was deleted
|
|
# after: 'My account is locked. Please unlock it.'
|
|
|
|
Dropping the request is silent and severe; leaving one boilerplate line behind is neither, so the
|
|
cleaning errs towards keeping text.
|
|
|
|
The same asymmetry drives the sign-off tests below. `_SIGNATURE_MARKERS` allowed a whole
|
|
sentence behind the closing word, so an ordinary body line beginning with one of them was read
|
|
as the start of a signature and everything after it was cut:
|
|
|
|
clean_email_body("Hi,\\n\\nThanks for the quick reply.\\nCould you refund invoice 4411?")
|
|
# before: 'Hi,' <- the request was cut away
|
|
|
|
The last block guards a different kind of silence. `email_questions` was defined twice, here and
|
|
in `laya/presets.py`, and `laya/__init__.py` re-exports the `presets` one. Editing the copy in
|
|
`laya/email.py` moved `laya.email.email_questions` and left `laya.email_questions` where it was,
|
|
with no test and no lint failing.
|
|
"""
|
|
import os
|
|
import sys
|
|
|
|
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
|
|
|
import laya # noqa: E402
|
|
from laya import email as email_module # noqa: E402
|
|
from laya import presets # noqa: E402
|
|
from laya.email import clean_email_body, email_state # noqa: E402
|
|
|
|
PASS, FAIL = [], []
|
|
|
|
|
|
def check(name, got, want):
|
|
if got == want:
|
|
PASS.append(name)
|
|
else:
|
|
FAIL.append("%s:\n got %r\n want %r" % (name, got, want))
|
|
|
|
|
|
def check_true(name, cond, detail=""):
|
|
if cond:
|
|
PASS.append(name)
|
|
else:
|
|
FAIL.append("%s %s" % (name, detail))
|
|
|
|
|
|
DISCLAIMER = "This email is confidential and intended solely for the named addressee."
|
|
|
|
# --------------------------------------------------------------- the request survives the footer
|
|
check(
|
|
"inline footer/no blank line keeps the request",
|
|
clean_email_body("My account is locked.\n%s\nPlease unlock it." % DISCLAIMER),
|
|
"My account is locked. Please unlock it.",
|
|
)
|
|
check(
|
|
"inline footer/unpunctuated request line is fully recovered",
|
|
clean_email_body("My account is locked\n%s\nPlease unlock it." % DISCLAIMER),
|
|
"My account is locked Please unlock it.",
|
|
)
|
|
check(
|
|
"inline footer/fused request without a trailing sentence",
|
|
clean_email_body("Please unlock my account\n%s" % DISCLAIMER),
|
|
"Please unlock my account",
|
|
)
|
|
check(
|
|
"inline footer/fused order reference is fully recovered",
|
|
clean_email_body("RMA 5521 is still pending\n%s\nPlease advise." % DISCLAIMER),
|
|
"RMA 5521 is still pending Please advise.",
|
|
)
|
|
check(
|
|
# Keep-ward trade-off, documented on purpose: a capitalised continuation of a
|
|
# boilerplate sentence can be a real fragment ("...in error,\nPlease delete it."),
|
|
# so it stays. Leaving one boilerplate line behind is harmless; dropping the
|
|
# request is not.
|
|
"inline footer/capitalised continuation stays",
|
|
clean_email_body("If you have received this message in error,\nPlease delete it."),
|
|
"Please delete it.",
|
|
)
|
|
check(
|
|
# Documented boundary: an all-lowercase fused request ("locked\nthis email ...")
|
|
# is indistinguishable from a wrapped boilerplate footer, so the first line is
|
|
# still lost. Only a newline followed by an uppercase letter splits.
|
|
"inline footer/lowercase fusion still recovers the tail",
|
|
clean_email_body(
|
|
"my account is locked\n"
|
|
"this email is confidential and intended solely for the named addressee.\n"
|
|
"please unlock it."
|
|
),
|
|
"please unlock it.",
|
|
)
|
|
check(
|
|
"inline footer/body is never emptied",
|
|
clean_email_body("My account is locked. %s" % DISCLAIMER),
|
|
"My account is locked.",
|
|
)
|
|
check_true(
|
|
"inline footer/is not empty",
|
|
clean_email_body("My account is locked. %s" % DISCLAIMER).strip() != "",
|
|
)
|
|
check(
|
|
"email_state/body keeps the request",
|
|
email_state("Locked out", "My account is locked. %s" % DISCLAIMER)["body"],
|
|
"My account is locked.",
|
|
)
|
|
|
|
# --------------------------------------------------------------- a pure footer is still removed
|
|
check(
|
|
"standalone footer paragraph is still dropped",
|
|
clean_email_body("My account is locked.\n\n%s" % DISCLAIMER),
|
|
"My account is locked.",
|
|
)
|
|
check(
|
|
"wrapped standalone footer is still dropped",
|
|
clean_email_body(
|
|
"My account is locked.\n\nThis email and any files transmitted with it are\n"
|
|
"confidential and intended solely for the named addressee."
|
|
),
|
|
"My account is locked.",
|
|
)
|
|
check(
|
|
"received-in-error footer is still dropped",
|
|
clean_email_body(
|
|
"Please reopen ticket 4411.\n\nIf you have received this message in error, delete it."
|
|
),
|
|
"Please reopen ticket 4411.",
|
|
)
|
|
|
|
# --------------------------------------------------------------- unrelated cleaning is unchanged
|
|
check(
|
|
"quoted history is still removed",
|
|
clean_email_body("Thanks for the update.\nOn Mon, Sep 20, Bob wrote:\n> original text"),
|
|
"Thanks for the update.",
|
|
)
|
|
check(
|
|
"signature block is still removed",
|
|
clean_email_body("Hi team,\nCan you confirm the refund?\nRegards,\nAlice"),
|
|
"Hi team,\nCan you confirm the refund?",
|
|
)
|
|
check("empty body stays empty", clean_email_body(""), "")
|
|
|
|
# --------------------------------------------------------------- Portuguese and Spanish mail
|
|
# With English-only markers none of this was removed, and the quoted history below (a cancellation)
|
|
# reached the model next to the new message (a refund request).
|
|
PT_EMAIL = """Olá equipe,
|
|
|
|
Fomos cobrados duas vezes na fatura de março. Por favor, estornem a cobrança duplicada hoje.
|
|
|
|
Atenciosamente,
|
|
João Silva
|
|
Financeiro - ACME Ltda
|
|
|
|
Enviado do meu iPhone
|
|
|
|
Esta mensagem pode conter informações confidenciais. Se você recebeu esta mensagem por engano, favor apagá-la.
|
|
|
|
Em seg., 22 de set. de 2026 às 10:14, Suporte <[email protected]> escreveu:
|
|
> Olá João, recebemos seu chamado de cancelamento do plano Enterprise.
|
|
"""
|
|
check(
|
|
"pt/full reply keeps only the request",
|
|
clean_email_body(PT_EMAIL),
|
|
"Olá equipe,\n\nFomos cobrados duas vezes na fatura de março. "
|
|
"Por favor, estornem a cobrança duplicada hoje.",
|
|
)
|
|
check(
|
|
"pt/outlook original-message block is cut",
|
|
clean_email_body(
|
|
"Segue o comprovante do pagamento.\n\n-----Mensagem original-----\n"
|
|
"De: Maria <[email protected]>\nAssunto: cancelar contrato\nQueremos cancelar o contrato."
|
|
),
|
|
"Segue o comprovante do pagamento.",
|
|
)
|
|
check(
|
|
"pt/outlook header without separator is cut",
|
|
clean_email_body(
|
|
"Segue o comprovante.\n\nDe: Maria <[email protected]>\nEnviado: segunda-feira\n"
|
|
"Assunto: cancelar contrato\nQueremos cancelar o contrato."
|
|
),
|
|
"Segue o comprovante.",
|
|
)
|
|
check(
|
|
"pt/gmail attribution wrapped over two lines is cut whole",
|
|
clean_email_body(
|
|
"O acesso voltou, obrigado.\n\nEm seg., 22 de set. de 2026 às 10:14, Suporte Técnico <\n"
|
|
"[email protected]> escreveu:\n> texto antigo"
|
|
),
|
|
"O acesso voltou, obrigado.",
|
|
)
|
|
check(
|
|
"pt/short sign-off is removed",
|
|
clean_email_body("Bom dia,\nO boleto de março não chegou.\nObrigado,\nAna"),
|
|
"Bom dia,\nO boleto de março não chegou.",
|
|
)
|
|
check(
|
|
"es/reply keeps only the request",
|
|
clean_email_body(
|
|
"Hola,\nNo puedo acceder a mi cuenta desde ayer.\nSaludos,\nCarlos\n\n"
|
|
"El lun, 22 sept 2026 a las 10:14, Soporte <[email protected]> escribió:\n> texto anterior"
|
|
),
|
|
"Hola,\nNo puedo acceder a mi cuenta desde ayer.",
|
|
)
|
|
check(
|
|
"es/disclaimer footer is dropped",
|
|
clean_email_body(
|
|
"Necesito la factura de marzo.\n\nSi usted ha recibido este mensaje por error, bórrelo."
|
|
),
|
|
"Necesito la factura de marzo.",
|
|
)
|
|
|
|
# --------------------------------------------------------------- ...without eating the request
|
|
check(
|
|
"pt/request mentioning `confidencial` is kept",
|
|
clean_email_body("Preciso do contrato confidencial assinado até sexta."),
|
|
"Preciso do contrato confidencial assinado até sexta.",
|
|
)
|
|
check(
|
|
"pt/`Obrigado` opening a sentence is not a signature",
|
|
clean_email_body("Oi,\nRecebi a resposta.\nObrigado pelo retorno, mas continua\nsem funcionar."),
|
|
"Oi,\nRecebi a resposta.\nObrigado pelo retorno, mas continua\nsem funcionar.",
|
|
)
|
|
check(
|
|
"pt/`caso tenha recebido` footer is dropped",
|
|
clean_email_body(
|
|
"Favor reenviar a nota fiscal.\n\nCaso tenha recebido esta mensagem por engano, "
|
|
"notifique o remetente."
|
|
),
|
|
"Favor reenviar a nota fiscal.",
|
|
)
|
|
check(
|
|
"pt/gmail attribution wrapped inside the name is cut whole",
|
|
clean_email_body(
|
|
"Resolvido, pode fechar.\n\nEm qua., 24 de set. de 2026 às 09:02, Suporte\n"
|
|
"Técnico <[email protected]> escreveu:\n> texto antigo"
|
|
),
|
|
"Resolvido, pode fechar.",
|
|
)
|
|
check(
|
|
"en/wrapped attribution is cut whole too",
|
|
clean_email_body(
|
|
"Fixed, thanks.\n\nOn Wed, Sep 24, 2026 at 9:02 AM Support Team <\n"
|
|
"[email protected]> wrote:\n> old text"
|
|
),
|
|
"Fixed, thanks.",
|
|
)
|
|
check(
|
|
"pt/`Em ... escreveu:` without a date is body text",
|
|
clean_email_body("Oi,\nEm resposta ao que você escreveu:\no pedido 4411 ainda não chegou."),
|
|
"Oi,\nEm resposta ao que você escreveu:\no pedido 4411 ainda não chegou.",
|
|
)
|
|
check(
|
|
"pt/`destinado exclusivamente` in a request is kept",
|
|
clean_email_body("O valor é destinado exclusivamente ao pagamento do boleto. Podem confirmar?"),
|
|
"O valor é destinado exclusivamente ao pagamento do boleto. Podem confirmar?",
|
|
)
|
|
check(
|
|
"pt/`De:` without an address is body text",
|
|
clean_email_body("Preciso das férias.\nDe: 10/09 a 15/09\nPode aprovar?"),
|
|
"Preciso das férias.\nDe: 10/09 a 15/09\nPode aprovar?",
|
|
)
|
|
|
|
# ------------------------------------------- `From:` starts prose, not only a quote header (#338)
|
|
# The marker used to be `^\s*From:\s.+$`, which matched any line beginning "From: ". English
|
|
# prose opens that way ("From: my side the integration works, but please refund ..."), so the
|
|
# request was deleted and the model answered an empty body. A reply header always carries the
|
|
# sender, so the marker is gated on an address -- the rule `De:` above already uses, for the
|
|
# same reason. The suite had no `From:` case at all before this, which is why it stayed green.
|
|
for label, body in [
|
|
("the reported case",
|
|
"Hi team,\nFrom: my side the integration works, but please refund the duplicate charge today.\nThanks"),
|
|
("lowercase", "Hi,\nfrom: my side the integration works, but please refund the charge.\nThanks"),
|
|
("uppercase", "Hi,\nFROM: my side the integration works, but please refund the charge.\nThanks"),
|
|
("extra spacing", "Hi,\nFrom: my side the integration works, but please refund the charge.\nThanks"),
|
|
("mid-body", "Hello,\nFrom: what I can see the charge was taken twice, please refund it.\nRegards"),
|
|
]:
|
|
check_true("en/`From:` prose keeps the request: " + label,
|
|
"refund" in clean_email_body(body), clean_email_body(body))
|
|
|
|
# The positive controls: a real quoted header block still goes, and takes its quoted request
|
|
# with it, so the fix does not simply stop cutting on `From:`.
|
|
for label, body in [
|
|
("address then Sent:",
|
|
"Hi,\nPlease look at this.\n\nFrom: Alice <[email protected]>\nSent: Monday\n"
|
|
"To: Bob\nSubject: Refund\n\nPlease refund the duplicate charge to my card."),
|
|
("address only", "Hi,\nSee below.\n\nFrom: [email protected]\nPlease refund the duplicate charge."),
|
|
("angle address", "Hi,\nSee below.\n\nFrom: <[email protected]>\nPlease refund the duplicate charge."),
|
|
("name and address",
|
|
"Hi,\nSee below.\n\nFrom: Alice Smith <[email protected]>\nPlease refund the duplicate charge."),
|
|
]:
|
|
check("en/quoted block still cut: " + label, clean_email_body(body), "Hi,\nPlease look at this."
|
|
if label == "address then Sent:" else "Hi,\nSee below.")
|
|
|
|
# ...and the request in the new part is kept even when a quoted block follows it.
|
|
check(
|
|
"en/`From:` quote header keeps the new request",
|
|
clean_email_body("Hi,\nPlease refund the duplicate charge today.\n\n"
|
|
"From: Alice <[email protected]>\nSent: Monday\n\nOld thread: refund the first charge."),
|
|
"Hi,\nPlease refund the duplicate charge today.",
|
|
)
|
|
|
|
# A bare `From: Name` header, with no address, is the English counterpart of `De: Maria Souza`:
|
|
# the marker above cannot cut it (that is the prose case), so it is recognised by its neighbours
|
|
# instead. Before this, only the Portuguese/Spanish `Enviado:`/`Data:` pair was a neighbour, so an
|
|
# English quote header without an address leaked its whole quoted block into the body.
|
|
for label, body in [
|
|
("then Sent:", "Hi,\nSee below.\n\nFrom: Alice Smith\nSent: Monday, 21 Sep 2026\n\nPlease refund."),
|
|
("then Date:", "Hi,\nSee below.\n\nFrom: Alice Smith\nDate: 21/09/2026\n\nPlease refund."),
|
|
("one-word name, then Sent:", "Hi,\nSee below.\n\nFrom: Alice\nSent: Monday\n\nPlease refund."),
|
|
]:
|
|
check("en/bare `From:` header still cut: " + label, clean_email_body(body), "Hi,\nSee below.")
|
|
|
|
# A bare `From:` with no header neighbour is not distinguishable from prose, and a dropped
|
|
# request is the worse error, so it is kept.
|
|
for label, body in [
|
|
("From: Name then To:", "Hi,\nSee below.\n\nFrom: Alice Smith\nTo: Bob\n\nPlease refund the charge."),
|
|
("From: Name alone", "Hi,\nSee below.\n\nFrom: Alice Smith\nPlease refund the charge."),
|
|
]:
|
|
check_true("en/bare `From:` with no header neighbour is kept: " + label,
|
|
"refund" in clean_email_body(body), clean_email_body(body))
|
|
|
|
# --------------------------------------------------------------- Brazilian clients and footers
|
|
BOLETO = "Preciso da segunda via do boleto."
|
|
for label, footer in [
|
|
("galaxy", "Enviado do meu Galaxy"),
|
|
("samsung default, 41 characters", "Enviado do meu smartphone Samsung Galaxy."),
|
|
("outlook ios", "Enviado do Outlook para iOS"),
|
|
("outlook android download line", "Obter o Outlook para Android"),
|
|
("windows mail", "Enviado do Email para Windows"),
|
|
("yahoo", "Enviado do Yahoo Mail no Android"),
|
|
("eco footer", "Antes de imprimir, pense em sua responsabilidade e compromisso com o MEIO AMBIENTE."),
|
|
("eco footer, reversed", "Pense no meio ambiente antes de imprimir este e-mail."),
|
|
]:
|
|
check("pt/footer removed: " + label, clean_email_body(BOLETO + "\n\n" + footer), BOLETO)
|
|
check(
|
|
"en/outlook download line removed",
|
|
clean_email_body("Please resend the invoice.\n\nGet Outlook for iOS"),
|
|
"Please resend the invoice.",
|
|
)
|
|
# Exchange leaves the address out of the header; the `Enviado:`/dated `Data:` line under it still marks it
|
|
for label, second in [("enviado", "Enviado: sexta-feira, 19 de setembro de 2026 10:02"),
|
|
("data", "Data: sexta-feira, 19 de setembro de 2026 10:02")]:
|
|
check(
|
|
"pt/outlook header without address is cut: " + label,
|
|
clean_email_body("Segue o comprovante.\n\nDe: Maria Souza\n%s\nPara: Suporte\n"
|
|
"Assunto: cancelar contrato\n\nQueremos cancelar o contrato." % second),
|
|
"Segue o comprovante.",
|
|
)
|
|
# ...and none of them may take the request with it
|
|
for label, body in [
|
|
("device words inside a request", "Oi,\nSegue o pedido.\nEnviado do meu celular o comprovante ontem."),
|
|
("`De:`/`Para:` date range", "Preciso das férias.\nDe: 10/09\nPara: 15/09\nPode aprovar?"),
|
|
("`De:` + undated `Data:`", "Relatório do evento.\nDe: João\nData: amanhã cedo\nPode confirmar?"),
|
|
("`antes de imprimir` in a request", "Antes de imprimir o boleto, confira o valor. Está errado."),
|
|
("`get` + device word", "Hi,\nThe box is at the front desk.\nGet mail"),
|
|
]:
|
|
check("pt/kept: " + label, clean_email_body(body), body)
|
|
|
|
# ------------------------------------------- a sign-off word inside the body is not a sign-off
|
|
SHORT = "Hi,\n\nThanks for the quick reply.\nCould you refund invoice 4411 as well?"
|
|
check("signoff word/short mail keeps the request", clean_email_body(SHORT), SHORT)
|
|
check(
|
|
"signoff word/thanks mid-body keeps what follows",
|
|
clean_email_body(
|
|
"Hello,\n\nWe were billed twice in March.\nThanks for looking into it.\n"
|
|
"The duplicate is 49 EUR on invoice 4411."
|
|
),
|
|
"Hello,\n\nWe were billed twice in March.\nThanks for looking into it.\n"
|
|
"The duplicate is 49 EUR on invoice 4411.",
|
|
)
|
|
check(
|
|
"signoff word/best mid-body keeps what follows",
|
|
clean_email_body(
|
|
"Hi team,\n\nOur account is locked.\nBest practice would be a manual unlock.\n"
|
|
"Please unlock account 88213 today."
|
|
),
|
|
"Hi team,\n\nOur account is locked.\nBest practice would be a manual unlock.\n"
|
|
"Please unlock account 88213 today.",
|
|
)
|
|
check(
|
|
"signoff word/email_state keeps the request",
|
|
email_state("Duplicate charge", SHORT)["body"],
|
|
SHORT,
|
|
)
|
|
|
|
# ------------------------------------------- real sign-offs are still cut (positive controls)
|
|
BODY = "Hi,\n\nPlease refund invoice 4411."
|
|
for label, tail in [
|
|
("thanks comma", "Thanks,\nAnna"),
|
|
("thanks bang", "Thanks!"),
|
|
("best regards", "Best regards,\nAnna"),
|
|
("kind regards", "Kind regards"),
|
|
("cheers name", "Cheers, Anna"),
|
|
("many thanks", "Many thanks,\nAnna Meier"),
|
|
("thank you", "Thank you,"),
|
|
("thanks in advance", "Thanks in advance,"),
|
|
("sincerely", "Sincerely,\nA. Meier"),
|
|
("sent from phone", "Sent from my iPhone"),
|
|
("dash delimiter", "--\nAnna Meier\nSupport"),
|
|
]:
|
|
check("signoff cut/" + label, clean_email_body("%s\n\n%s" % (BODY, tail)), BODY)
|
|
|
|
# ------------------------------------------- closings the case rule did not reach (#132 follow-up)
|
|
# These were cut before #132 and are not now: `warmest` is not in the alternation, `and regards`
|
|
# is not one of its continuations, and `[A-Z]` is ASCII, so a name in any other script reads as a
|
|
# sentence. Each leaves the signature block in the body that the sign-off rule exists to remove.
|
|
for label, tail in [
|
|
("thanks and regards", "Thanks and regards,\nAnna"),
|
|
("thanks & regards", "Thanks & Regards,\nAnna"),
|
|
("warmest regards", "Warmest regards,\nAnna"),
|
|
("warmest wishes", "Warmest wishes,"),
|
|
("non-ascii name", "Regards, Łukasz"),
|
|
("non-ascii name, accented", "Thanks, José"),
|
|
("cyrillic name", "Regards, Дмитрий"),
|
|
]:
|
|
check("signoff cut/" + label, clean_email_body("%s\n\n%s" % (BODY, tail)), BODY)
|
|
|
|
# ...and the wider closing must not swallow a sentence that merely starts the same way
|
|
for label, body in [
|
|
("and + sentence", "Hi,\n\nPlease refund 4411.\nThanks and the team will confirm it today."),
|
|
("warmest + sentence", "Hi,\n\nThe room is cold.\nWarmest setting still reads 18 degrees."),
|
|
]:
|
|
check("signoff kept/" + label, clean_email_body(body), body)
|
|
|
|
|
|
# ------------------------------------------------- the word, without the disclaimer
|
|
# `confidential` was a bare substring of `_DISCLAIMER`, so any sentence that merely
|
|
# mentioned it was dropped. A one-sentence body that mentions it was deleted whole and
|
|
# the model was then scored on an empty state, silently. The Portuguese branches beside
|
|
# it were already tied to disclaimer phrasing for this reason; English now is too.
|
|
for label, body in [
|
|
("a question about the word", "Is this confidential?"),
|
|
("a policy question", "What is your confidentiality policy?"),
|
|
("a request containing the word", "Please keep this confidential but process my refund."),
|
|
("a request about handling", "Please treat this as confidential."),
|
|
("a question with a dash", "This is confidential - can you help?"),
|
|
("a question about an attachment", "Is the attached document confidential?"),
|
|
("a label prefix", "Confidential: I need a refund."),
|
|
("a question about information", "What is the information policy for contractors?"),
|
|
]:
|
|
check("word only/kept: " + label, clean_email_body(body), body)
|
|
check_true(
|
|
"word only/body is never emptied",
|
|
clean_email_body("Is this confidential?").strip() != "",
|
|
)
|
|
check(
|
|
"word only/email_state keeps the request",
|
|
email_state("Question", "Is this confidential?")["body"],
|
|
"Is this confidential?",
|
|
)
|
|
|
|
# ...while the real footers those branches exist for are still dropped
|
|
for label, body in [
|
|
("named addressee", "This email is confidential and intended solely for the named addressee."),
|
|
("the individual addressed",
|
|
"This message is confidential and intended solely for the use of the individual to whom it is addressed."),
|
|
("may be privileged", "The information in this email is confidential and may be privileged."),
|
|
("wrapped across lines",
|
|
"This email and any files transmitted with it are\n"
|
|
"confidential and intended solely for the named addressee."),
|
|
]:
|
|
check_true("word only/still dropped: " + label, not clean_email_body(body).strip())
|
|
check(
|
|
"word only/request before a footer survives",
|
|
clean_email_body("My account is locked.\n"
|
|
"This email is confidential and intended solely for the named addressee.\n"
|
|
"Please unlock it."),
|
|
"My account is locked. Please unlock it.",
|
|
)
|
|
check(
|
|
"word only/request inside one sentence survives",
|
|
clean_email_body("Please unlock it. This email is confidential and intended solely "
|
|
"for the named addressee."),
|
|
"Please unlock it.",
|
|
)
|
|
|
|
|
|
# ------------------------------------------- email_questions has exactly one definition
|
|
check_true(
|
|
"email_questions/one definition behind both module paths",
|
|
email_module.email_questions is presets.email_questions,
|
|
"laya.email.email_questions is not laya.presets.email_questions",
|
|
)
|
|
check_true(
|
|
"email_questions/the package export is that same object",
|
|
laya.email_questions is presets.email_questions,
|
|
)
|
|
check(
|
|
"email_questions/both paths answer the same",
|
|
email_module.email_questions(),
|
|
laya.email_questions(),
|
|
)
|
|
check(
|
|
"email_questions/a caller override reaches both paths",
|
|
email_module.email_questions({"legal": "contracts"})["category"]["criteria"],
|
|
laya.email_questions({"legal": "contracts"})["category"]["criteria"],
|
|
)
|
|
|
|
|
|
print("\n%d passed, %d failed" % (len(PASS), len(FAIL)))
|
|
for f in FAIL:
|
|
print(" FAIL " + f)
|
|
sys.exit(1 if FAIL else 0)
|