Skip to content

Add LogSumExp functions #34

New issue

Have a question about this project? No Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “No Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? No Sign in to your account

Closed
delphidabbler opened this issue Jan 6, 2025 · 3 comments
Closed

Add LogSumExp functions #34

delphidabbler opened this issue Jan 6, 2025 · 3 comments
Assignees
Labels
completed Issue completed and committed to develop. To be closed on next release enhancement New feature or request

Comments

@delphidabbler
Copy link
Owner

LogSumExp

LogSumExp or LSE for a sequence x1,...,xn is is defined as the logarithm of the sum of the exponentials of the arguments:

LSE(x1,...,xn) = log(exp(x1) + ... + exp(xn))

Source: Wikipedia

function LSE(const A: array of Extended): Extended;
begin
  var Sum: Extended := 0.0;
  for var X in A do
    Sum := Sum + Exp(X):
  Result := Ln(Sum);
end;

Strictly Convex LogSumExp

Screenshot_20230720-040354_Chrome

Source: Wikipedia

Since exp(0) = e^0 = 1, we have

LSE(0, x1,...,xn) = log(exp(0) + exp(x1) + ... + exp(xn))
                  = log(1 + exp(x1) + ... + exp(xn)),

So, in Pascal we can have:

function ConvexLSE(const A: array of Extended): Extended;
begin
  var Sum: Extended := 1.0; // for additional Exp(0) term
  for var X in A do
    Sum := Sum + Exp(X):
  Result := Ln(Sum);
end;

This issue was extracted from issue #16

@delphidabbler delphidabbler self-assigned this Jan 6, 2025
@delphidabbler delphidabbler added enhancement New feature or request considering Issue is currently under consideration labels Jan 6, 2025
@github-project-automation github-project-automation bot moved this to Considering in Code Snippets Jan 6, 2025
@delphidabbler delphidabbler added accepted Issue will be actioned and removed considering Issue is currently under consideration labels Jan 10, 2025
@delphidabbler delphidabbler moved this from Considering to Accepted in Code Snippets Jan 10, 2025
@delphidabbler
Copy link
Owner Author

The Softmax function is closely related to LSE. This should also be implemented.

@delphidabbler
Copy link
Owner Author

Strictly Convex LogSumExp

Decided not to implement this.

@delphidabbler
Copy link
Owner Author

Implemented by merge commit 4186f6e

@delphidabbler delphidabbler added completed Issue completed and committed to develop. To be closed on next release and removed accepted Issue will be actioned labels Jan 10, 2025
@delphidabbler delphidabbler moved this from Accepted to Completed in Code Snippets Jan 10, 2025
@delphidabbler delphidabbler added this to the Next Release milestone Jan 10, 2025
@delphidabbler delphidabbler removed this from the Next Release milestone Jan 19, 2025
No Sign up for free to join this conversation on GitHub. Already have an account? No Sign in to comment
Labels
completed Issue completed and committed to develop. To be closed on next release enhancement New feature or request
Projects
Status: Completed
Development

No branches or pull requests

1 participant