Showing posts with label Domino. Show all posts
Showing posts with label Domino. Show all posts

Thursday, November 02, 2023

Configuring Entitlement Tracking in Domino 12

In the realm of HCL Domino Server 12.0, the feature of "Entitlement Tracking" has become a vital component for organizations.

While comprehensive information regarding Entitlement Tracking is available through HCL, I needed to know some practical management aspects, such as disabling the feature and adjusting intervals etc.

Disabling Entitlement Tracking:

To disable entitlement tracking, add the following entry and restart the server:

DISABLE_ENTITLEMENT_TRACKING=1

Debugging for Entitlement Tracking Issues:

Debug settings can be incredibly useful for troubleshooting any issues related to Entitlement Tracking. Here's how to configure debugging options:

DEBUG_ENTITLEMENT_AGGREGATOR_INTERVAL=60
DEBUG_UPDATE_ENTITLEMENT_TRACKING=2
ES_OPT_TIMING=1
DEBUG_DIRCAT=3

Hope that will help somebody.

Friday, October 27, 2023

Overcoming Domino's Agent Scheduling Limitations with JavaAddin

If you're frustrated with Domino's limitations on scheduling agents to run more frequently than once every 5 minutes, you're not alone. As a programmer, you understand the need for flexibility and control in your applications. In this article, we'll discuss a practical solution: creating a JavaAddin for Domino that can trigger agents at shorter intervals, allowing you to gain more fine-grained control over your scheduled tasks.

Understanding the Domino Agent Scheduler


Domino provides a robust environment for running scheduled agents. However, it imposes a minimum time gap of 5 minutes between consecutive runs of the same agent. This limitation can be a roadblock for applications that require more frequent execution.


The Power of JavaAddins


JavaAddins offer a way to extend Domino's functionality using Java code. This opens up a world of possibilities, including overcoming the 5-minute scheduling restriction. Here's how you can do it:

1. Setting Up Your JavaAddin

To get started, you'll need to create a JavaAddin. This involves writing Java code to interface with Domino. The code should enable you to trigger agents at shorter intervals than what Domino's native scheduling allows.

2. Utilizing Timers

One of the most effective ways to bypass the 5-minute limitation is to use timers in your JavaAddin. With timers, you can execute your agent at precise intervals, even down to seconds. Here's a simplified example of how this could look in your Java code:
import lotus.domino.*;
public class CustomScheduler extends JavaServerAddin {
    public void runNotes() {
        try {
            Session session = NotesFactory.createSession();
            Database database = session.getDatabase("", "YourDatabase.nsf");
            Agent agent = database.getAgent("YourAgent");

            // Set the execution interval in milliseconds
            int interval = 30000; // 30 seconds

            while (true) {
                agent.runWithDocumentContext(null);
                sleep(interval);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}
This code defines a Java thread that runs your specified agent every 30 seconds, effectively bypassing Domino's 5-minute restriction.

3. Deploying Your JavaAddin

Once you've created your JavaAddin, you need to deploy it within your Domino environment. Ensure that the necessary permissions and access controls are in place.

4. Monitoring and Maintenance

Regularly monitor the execution of your custom scheduling solution. Ensure that it's working as expected and doesn't place undue stress on your Domino server.


Conclusion


By creating a JavaAddin for Domino that can trigger agents more frequently, you can take control of your scheduling needs. This solution empowers you to run your agents at shorter intervals, achieving the level of precision your applications require. While this approach requires some development effort, the benefits of fine-grained agent scheduling can greatly enhance your Domino-based applications.

In summary, if you're tired of being constrained by Domino's 5-minute scheduling limitation, consider the power of JavaAddins to break free and gain control over your scheduled agents.

I have also created a bit more advanced setup which you can get on github: DominoAgentsHelper 

Stay tuned for more technical insights, delivered directly to the point, in future articles.

Thursday, September 08, 2022

Java Freemarker with Domino

There are plenty of different Java template engines but for last years I used to stick to FreeMarker. It's open sourced and licensed under the Apache License, Version 2.0.

Here I only want to demonstrate how to integrate it with Domino nicely as it requires to write TemplateLoader class.

Build result based on tempalte "page"
Configuration cfg = new Configuration(Configuration.VERSION_2_3_31);
cfg.setDefaultEncoding("UTF-8");
cfg.setTemplateExceptionHandler(TemplateExceptionHandler.RETHROW_HANDLER);
cfg.setLogTemplateExceptions(false);

Get the template (uses cache internally)
DominoTemplateLoader dominoLoader = new DominoTemplateLoader(getDatabase());
cfg.setTemplateLoader(dominoLoader);

Template template = cfg.getTemplate("page");

/* Merge data-model with template */
HashMap tags = new HashMap();
tags.put("title", "hellow world");
tags.put("description", "Neque porro quisquam est qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit.");
			
Writer out = new StringWriter();
template.process(tags, out);
String html = out.toString();
The most important part was actually to build DominoTemplateLoader class and below you can see it
public class DominoTemplateLoader implements TemplateLoader {
	private View m_view;

	public DominoTemplateLoader(Database database) throws NotesException {
		m_view = database.getView("($Template)");
	}

	public void closeTemplateSource(Object templateSource) throws IOException {
		Document doc = (Document) templateSource;
		try {
			doc.recycle();
		} catch (NotesException e) {
			e.printStackTrace();
		}
	}

	public Object findTemplateSource(String id) throws IOException {
		try {
			return m_view.getDocumentByKey(id, true);
		} catch (NotesException e) {
			e.printStackTrace();
		}

		return null;
	}

	public long getLastModified(Object templateSource) {
		Document doc = (Document) templateSource;

		try {
			return doc.getLastModified().toJavaDate().getTime();
		} catch (NotesException e) {
			e.printStackTrace();
		}

		return 0;
	}

	public Reader getReader(Object templateSource, String encoding) throws IOException {
		if (templateSource == null) return null;
		
		Document doc = (Document) templateSource;
		try {
			return doc.getFirstItem("Body").getReader();
		} catch (NotesException e) {
			e.printStackTrace();
		}
		return null;
	}
}
As you can see the Loader class get document form a view and simply get data from item Body.

Friday, December 10, 2021

Journalize email from Exchange to Domino using Addin

One of my customer moves to Office 365 and also wants to move to to Exchange/Outlook 356 during next year while keeping Domino app running as it is of now.

They have customized mail boxes with few actions which allow to journalize emails into their Domino applications and that is quite critical functionality so that would need to be mirrored.

We have decided to built Outlook Add-in (works in web, client and also with mobile devices).

Here are a few advises to those who would need to developer similar functionality.

1. Create a outlook add-in project and define manifest

You would have to create a project with manifest and needed html, css, js elementets. You can easily find information how to do that on MS sites (not going to provide any links as they could change in future). That will allow you to define UI, see my example.


2. Send memo ID to Domino

We need to get information about email from Outlook Add in and send those items to Domino (you would have to build REST API on Domino side that can receive data from Outlook).

Office.context.mailbox.getCallbackTokenAsync(function(result) {

  ...

  var token = result.value;
  var ewsurl = Office.context.mailbox.restUrl;
  var ewsItemId = Office.context.mailbox.item.itemId;
  const itemId = Office.context.mailbox.convertToRestId(ewsItemId,Office.MailboxEnums.RestVersion.v2_0);

  // send token, ewsurl and itemId to Domino endpoint
  ...

}

Having those keys (token url and itemId) you can pull email in Mime format

3. Convert Mime to Notes email

So at this point Domino received data from Add in and can finally do another request to Exchange server (using token, ewsurl and itemId) to read the memo MIME

Dim http As NotesHTTPRequest
Dim enpoint As string
		
Set http = m_app.NotesSession.Createhttprequest()
Call http.Setheaderfield("Authorization", "Bearer " + token)
enpoint = ewsurl + |/v2.0/me/messages/| + itemId + |/$value|
		
getItemMIME = http.get(enpoint)

There is no native Domino LS/Java Mime Parser however I found working example by Stephan: Importing EML files into Notes (lots of them). It worked well, but seems it does not handle inline images (need to do more testing etc).

Alternatively I was told that there is MimeMessageParser.java that writes MIME to a Notes document. This class is part of the XPages Extension Library. So it has sense to compare them.

Tuesday, October 20, 2020

JavaServerAddin in Domino - constructor and schedule

In this article we will improve our java addin 'DemoAddin' with following features:

  1. Constructor that accepts parameters when we load java addin from console.
  2. We will schedule output to console amount of registered person in names.nsf (every 33 seconds).
  3. Define destructor.
  4. Load (with parameter) and Unload addin from console

Constructor

We will define 2 constructor, one that can accept parameters and one in case if we load addin without any parameters. It is pretty obvious how it works.

// we expect our first parameter is dedicated for secondsElapsed
public DemoAddin(String[] args) {
	this.secondsElapsed = Integer.parseInt(args[0]);
}

// constructor if no parameters
public DemoAddin() {}

Means if we run command like below it will run using constructor with parameters

load runjava org.demo.DemoAddin 33

Schedule worker

There is a method (it works but deprecated, however I have not found what should be use instead). The snippet below run constant loop, open names.nsf, read amount of users in the view People and output it to console. The main line here is this.addInRunning(), it keeps loop running forever (until we change it to this.stopAddin() or unload addin from console)

session = NotesFactory.createSession();
String server = session.getServerName();

while (this.addInRunning()) {
	/* gives control to other task in non preemptive os*/
	OSPreemptOccasionally();

	if (this.AddInHasSecondsElapsed(secondsElapsed)) {
		ab = session.getDatabase(server, "names.nsf");
		long count = ab.getView("People").getAllEntries().getCount();
		logMessage("Count of persons: " + Long.toString(count));
		ab.recycle();
	}
}

Destructor

Keep in mind that we need to be careful with Notes object, we have to release memory (recycle) after we no longer use them. So it's a good idea to create own terminate method that release memory for all Notes object you delcared and use it when addin unloads. There is also built-in method finalize so you can put code there, but I prefer to have own method and use it in places I need

private void terminate() {
	try {
		if (this.ab != null) {
			this.ab.recycle();
		}
		if (this.session != null) {
			this.session.recycle();
		}

		logMessage("UNLOADED (OK)");
	} catch (NotesException e) {
		logMessage("UNLOADED (**FAILED**)");
	}
}

Load (with parameter) and Unload addin from console

In order to run addin with parameter simply add it after name of Addin, use space as a separator when you need more than 1 parameter

load runjava org.demo.DemoAddin 33
[1098:0002-23A0] 10/20/2020 11:15:00 AM  JVM: Java Virtual Machine initialized.
[1098:0002-23A0] 10/20/2020 11:15:00 AM  RunJava: Started org/demo/DemoAddin Java task.
[1098:0004-3984] 10/20/2020 11:15:00 AM  DemoAddin: version             2
[1098:0004-3984] 10/20/2020 11:15:00 AM  DemoAddin: build date          2020-10-19 11:00 CET
[1098:0004-3984] 10/20/2020 11:15:00 AM  DemoAddin: java                1.8
[1098:0004-3984] 10/20/2020 11:15:00 AM  DemoAddin: seconds elapsed     33
[1098:0004-3984] 10/20/2020 11:15:33 AM  DemoAddin: Count of persons: 11
[1098:0004-3984] 10/20/2020 11:16:06 AM  DemoAddin: Count of persons: 11
[1098:0004-3984] 10/20/2020 11:16:39 AM  DemoAddin: Count of persons: 11
[1098:0004-3984] 10/20/2020 11:17:12 AM  DemoAddin: Count of persons: 11

When you want to unload addin using console here is a command

tell runjava unload org.demo.DemoAddin

And you should be see confirmation on console if everything went fine

[1098:0004-3984] 10/20/2020 11:26:55 AM  DemoAddin: UNLOADED (OK)
[1098:0002-23A0] 10/20/2020 11:26:55 AM  RunJava: Finalized org/demo/DemoAddin Java task.
[1098:0002-23A0] 10/20/2020 11:26:56 AM  RunJava shutdown.

If you are interested in this topic I can recommend at least two more sources NSFTools.com JavaAddinTest and AndyBrunner / Domino-JAddin or wait for new articles in my blog :-). Also feel free to ask questions if you are uncertain.

Full version of DemoAddin class is hosted on github: DominoDemoAddin

All articles in series
  1. JavaServerAddin in Domino - introduction
  2. JavaServerAddin in Domino - constructor and schedule

Wednesday, October 14, 2020

JavaServerAddin in Domino - introduction

I will show how to build, register and load simple JavaAddin for Domino. I'm not entirely sure if lotus.notes.addins.JavaServerAddin is supported by HCL, so use that for your own sake.

1) Java class

import lotus.notes.addins.JavaServerAddin;

public class DemoAddin extends JavaServerAddin {
	public void runNotes() {
		AddInLogMessageText("Hello world", 0);
	}
}

2) JAR - from project

Export/build JAR file from the DemoAddin project (we are going to put jar file in the Domino folder).

3) Register JavaAddin

Place JAR file under Domino, f.x. path could be (DemoAddin is a folder and it could be just any name, DemoAddin-1.jar is our JAR file we built earlier)

C:\IBM\Domino\DemoAddin\DemoAddin-1.jar

and then register it in server's notes.ini using variable JAVAUSERCLASSES. In case if there are other addin registered there use semicolon as a separator for Windows and a colon for Linux

JAVAUSERCLASSES=addin-1;.\DemoAddin\DemoAddin-1.jar;addin-2

Alternatively simply put JAR file into the folder \jvm\lib\ext, but personally I prefer to keep customization separately instead of mixing core JAR files with customization. Additionally I'm not sure what happens to custom JAR file when is upgradet.

4) Load JavaAddin

It's time to run our DemoAddin. From console run following command

load runjava DemoAddin

Take into account if your include your class into a package, f.x. package org.demo; than you should add that into run command

load runjava org.demo.DemoAddin

If everything went fine you should see 3 lines

RunJava: Started DemoAddin Java task.
Hello world
RunJava: Finalized DemoAddin Java task.

Possible error

If you registered JAR file incorrectly, the error could be like this. In such case just make sure you followed steps properly.

RunJava: Can't find class DemoAddin1 or lotus/notes/addins/demoaddin1/DemoAddin1 in the classpath.  Class names are case-sensitive.

If I find time, I will make few more posts about this topic. It's really just a tip of the iceberg.

All articles in series
  1. JavaServerAddin in Domino - introduction
  2. JavaServerAddin in Domino - constructor and schedule

Tuesday, July 17, 2018

Domino with Java 6 and TSL 1.2

Recently I have faced an issue where one of our provider changed SSL and they disabled supporting of TLS 1.0 (as far as I understand it's non secure ourdays) and TLS 1.2 should be used instead. As a result our java agents (which used HttpsURLConnection) could not connect anymore to provider.

Error message looked like this:
Caused by: java.security.AccessControlException: Access denied (javax.net.ssl.SSLPermission setHostnameVerifier)
I have found 2 possible solutions:

Enable TLS 1.2 on Domino (applicable only for 9.0.1 FP3 IF2 and higher)


The Domino JVM is based on Java 1.6 and default settings configured in a way to use TLS 1.0. Luckily our Domino servers had version 9.0.1 FP4 (and TSL 1.2 support has been added since FP3 IF2). So our version was capable to work with 1.2 (in theory) but it took some time to make it work.

In order to configure Domino JVM to use TLS 1.2 you need to:
  1. Create JVM settings file, f.x. C:\Domino\jvmOptions.ini
  2. Add parameter in jvmOptions.ini
    https.protocols=TLSv1.2
  3. Add path to jvmOptions.ini file in notes.ini
    JavaUserOptionsFile=C:\Domino\jvmOptions.ini
After you added settings don't forget to restart Domino server. Keep in mind that setting is global meaning all agents that will start to use TLS1.2 therefore it is definitely worth to verify everything before and after this fix.

Java library solution


If that is not a way you can go with (f.x. Domino has lower version or something won'f work if you switch to TLS 1.2) then it's still possible to make custom Java Library that will make it possible, see link: How to use TLS 1.2 in Java 6.

It worked for me as well, but it requires to give permission in java policy on Domino server.

Monday, June 15, 2015

Insufficient memory - NSF pool is full

Recently I've faced up with an odd issue. When users tried to open Lotus Notes applications they got a warning saying
Insufficient memory - NSF pool is full

What 'Insufficient memory - NSF pool is full' is about?

As far as I understood it is memory leak on IBM Domino server. There are different kind of issues similar to this one:
  • Insufficient memory - NSF pool is full
  • Insufficient memory - index pool is full
  • Insufficient memory - NSF directory manager pool is full
  • Insufficient memory - Event pool is full
  • Insufficient memory - NSF Monitor pool is full
  • Insufficient memory - Network pool is full

Fixing the issue

Of course it require investigation why it happened but as a temporary fix you can simple increase memory while you are looking into the problem (if you do so :)). The variable name responsible for this pool is NSF_Buffer_Pool_Size (number of bytes) or alternatively you can use NSF_Buffer_Pool_Size_MB (number of megabytes). I'm not going to explain details about this variable, since you can find everything on IBM, just open this link: NSF_Buffer_Pool_Size

Adding or changing NSF_Buffer_Pool_Size_MB in notes.ini

I didn't find a default size for NSF_Buffer_Pool_Size (One article mentioned 32MB as a default value, however I could not really confirm that with IBM documentations). Anyway, for I set 64MB and from now I will keep my eyes open on this problem.
Let's change notes.ini using a Configuration Settings document
  1. First, open IBM Domino Administrator and find a Configuration tab there.
  2. Find Configuration settings now (in R9, it's nuder Server\Configurations), if it is not there, you may need to create it first.
  3. Highlight right server and and click button Edit Configuration and switch to NOTES.INI Settings tab.
  4. There must be a button: Set/Modify Parameters at the bottom, click on it and add/change your variable.
  5. Save configuration and Restart Server.

Done.

Thursday, March 20, 2014

Getting mail.box from Domino server

Domino allows to setup up to 10 mail boxes for 1 server. By default it 1 mail box with name mail.box however if you increase number of mail boxes to 2 and more Domino will create mail1.box, mail2.box ... and you must remember about it.
In order to get mail box without doing lookup to ServerConfig document, just try to initiate mail1.box first and if it does not exists go for mail.box
public Database getMailBox(Session session) throws NotesException{
 String server = session.getServerName();
 Database mailbox = session.getDatabase(server, "mail1.box");
 if (!mailbox.isOpen()) {
  mailbox = session.getDatabase(server, "mail.box");
 }
 return mailbox;
}
There is topic on how to force Domino to use mail.box when multiple mail.boxes are enabled

Tuesday, January 28, 2014

Domino and No trusted certificate found

We have few agents that communicate with other systems via and today at 13:00 we got such answer from Domino when we tried to set connection with external system. That was a huge problem for us since it affect our business.
 javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.j: No trusted certificate found  
  at com.ibm.jsse2.o.a(o.java:8)  
  at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:549)  
  at com.ibm.jsse2.kb.a(kb.java:355)  
  at com.ibm.jsse2.kb.a(kb.java:130)  
  at com.ibm.jsse2.lb.a(lb.java:135)  
  at com.ibm.jsse2.lb.a(lb.java:368)  
  at com.ibm.jsse2.kb.s(kb.java:442)  
  at com.ibm.jsse2.kb.a(kb.java:136)  
  at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:495)  
  at com.ibm.jsse2.SSLSocketImpl.h(SSLSocketImpl.java:223)  
  at com.ibm.jsse2.SSLSocketImpl.a(SSLSocketImpl.java:724)  
  at com.ibm.jsse2.SSLSocketImpl.startHandshake(SSLSocketImpl.java:81)  
  at com.ibm.net.ssl.www2.protocol.https.c.afterConnect(c.java:8)  
  at com.ibm.net.ssl.www2.protocol.https.d.connect(d.java:20)  
It took some time to fix it, but here is a solution

Problem

A Java application running on a Domino server connecting over SSL to another server may require having the SSL certificate authority of the other server imported into its JVM.

Symptom

When a Java application running on a Domino server connects over SSL to another server, but does not have that server's trusted root certificates, an error may occur. One example of such an error is: HTTP JVM: javax.net.ssl.SSLHandshakeException: com.ibm.jsse2.util.g: No trusted certificate found.

Cause

The trusted root certificates that signed the remote server's SSL certificate must be also be trusted by the Domino server's JVM if a Java application is making an SSL connection.

Resolving the problem

To add the trusted root certificates to a Domino server JVM follow these steps:

A. Obtain the Certificate to be Imported

Each browser displays certificates in different ways, but they are usually quite similar. On the browser's URL bar, there is usually a zone that you can click on to display SSL certificate information. For example, you may see a padlock in the status bar, and clicking on the padlock opens the certificate information. Once the certificate information is open, click on the "Certification Path" informatino. There normally will be a way to export each of the signing certificates (trusted roots). Export the certifiers in the "Base-64 encoded X.509 (.CER)" format. The exported file in this format will be an ASCII text file that has "BEGIN CERTIFICATE" and "END CERTIFICATE" lines at the top and bottom. Once you have exported the certificates that signed the remote server's SSL certificate you can then import them into the JVM.

B. Import the SSL certifier into the JVM.

If Domino is on a UNIX server, perform these steps on a Windows workstation, and then move the cacerts to the server after the import is completed.
Import the SSL Certificate into the JVM using these steps:
  • Open a command line and change directory to C:\Lotus\Domino\jvm\bin.
  • Run the batch file "IKEYMAN.exe" (a Java application will load).
  • Click "Key Database File" then "Open".
  • Browse to C:\Lotus\Domino\jvm\lib\security\cacerts. Note, you will have to view "All Files" to locate cacerts.
  • Supply the default password of "changeit". Note, consult your administrator if you receive an error pertaining to the password.
  • Select "Signer Certificates" in the drop-down menu.
  • Click "Add"
  • Select "Browse" and locate the .CER file you copied.
  • Click "OK" and enter a descriptive label.
  • On the Domino console issue the command "restart task http".
Original link I found on IBM Importing an SSL Certificate Authority into the JVM

Related topics:
IBM Domino Java: No trusted certificate found. Fail?
Domino and No trusted certificate found
Disabling certificate validation in Java

Friday, October 18, 2013

Issues when developing IBM Domino Notes applications

Some minor stuff about Domino Designer.
  • Undo/'Ctrl+Z' does does not work properly when you developing in Lotus Script, if you do it few times it might mess your code
  • 'Ctrl+C'/'Ctrl+V' sometimes does not work from first attempt, so you need to do it twice
  • 'Ctrl+C'/'Ctrl+V' sometimes it sets focus on penultimate character so you have to do 1 extra actions to continue write code
  • I still have crashes in Domino Designer without any reasons and I've no customization to my Designer
  • I'm not happy about speed in Designer, however maybe I expect to much
It is 2013 and Ctrl+Z still work odd, how could it be?
Do you have something to add to that list? :) Please - comments about that!

Friday, August 09, 2013

How we cook GitHub, Jira and Jenkins with Domino

Why would I use it?

I'll describe briefly how we use GitHub, Jira and Jenkins together to make better development process. More details will come later when I get some desire :). OK, lets look what benefits we have from using these tools.

GitHub with Jenkins

GitHub has number of Service Hooks and one of them is for Jenkins. It allows to trigger build jobs when pushes are made to GitHub. You don't need to have any skill to do that, let's look on screen below. So once you made a push, Jenkins start to work.

GitHub with JIRA

It's possible to made connection between JIRA and GitHub, so once you made push to GitHub JIRA will update related ticket, you only need to specify ticket number in comments. As a result you will have history with commits directly in JIRA ticket. Let's look on ticket I've closed few days ago.
You can see what exactly have been done in order to fix the ticket. Also you can get more details if you wish, you only need to click on any of those updated files and then you will see that

Jenkins and Domino

That is most complicated part. In order to synchronize changes from GitHub to Domino application we use Jenkins (if you remember once push happens we trigger hook on Jenkins). Unfortunately there are no plugins that could do that, instead you have to write your own. That's it why we have ~60 seconds delay (I mentioned it in previous posts) after push.

Summary

For me JIRA and GitHub is de facto standard our days. Jenkins in our case is necessary to use in order to push changes from GitHub to Domino. It's a bit complicated process so you need before, if it is really what you need.

Other articles in this series

  1. How we build our web applications based on Domino
  2. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
  3. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
  4. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
  5. Using Git+Jira is must and Jenkins server as builder server
  6. Auto-tests.

Thursday, August 08, 2013

Java as backend for web applications based on Domino

Is it enough to use only Java?

Yes. Our newly created applications have been written only for web and we did all back-end with Java only (the only exclusions are selection for Views). No, we do not miss @Formula, Lotus Script or SSJS, we simply do not need those languages in our daily work. Java is better than LS/SSJS for us. You can develop much better/faster using Java and get all candies: open sources, forums, stackoverflow or even shift at some point to Java completely if it has sense for you. However, if you read my main posts I mentioned that we still had some problems while work with Java, Solution isn't really perfect and let me explain all those weakness we have right now.
  • Domino has 2 Java runtimes (huge pain for us): one is for xPages and another one for Java Libraries+Agents. Java Libraries can be reuse in xPages and back JAVA/JAR are not accessible from Libraies/Agents. Our Java libraries are core for xPages and it means we have to build JAR from them and include them for xPages. It takes time and we really do not like that, it simply looks wrong. The solution is to put JAR to Domino OS, however we are limited with access to Domino's OS. Due to that fact we have one serious disadvantage - our JAR files that we attached to Libraries and Agents have to be extracted and loaded to memory each time we run agents.
  • We are forced to use IBM Designer (and few of us would like to work in different IDE). As I mentioned we work via GitHub and it takes some time to build project (developer -> github -> jenkins -> domino). That's not a problem for FE, as they can do their tasks locally and after all just commit changes. BE part is located on Server so if we work without IBM Designer we have to wait 1-2 mins till changes come to it and only then we can do our tests. That's sad.

Does it mean @formula, Lotus Script, SSJS are not required to know?

Those languages are still important part of our 'old applications' and it will take years to completely re-write them to web and there should be a sense to do that, as it is time/money, @formula language is still only 1 way to make selection in Notes Views.

Pros
  • open source;
  • problems resolving (forums, community);
  • more abilities;
  • in case of migration to java platform, easy migration;
Cons
  • 2 Java Runtimes
  • ...

Summary

We are happy with what we have so far. We still have problems in terms of how to setup our development process. We will continue to look for perfect solution for our problems.

Other articles in this series

  1. How we build our web applications based on Domino
  2. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
  3. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
  4. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
  5. Using Git+Jira is must and Jenkins server as builder server
  6. Auto-tests.

Wednesday, August 07, 2013

Frontend developers without knowledge about Domino

Do FE developers must have understanding of IBM Domino/Notes platform?

I believe they should know we use Domino as server, however that's it. Domino is server, so it is responsibility of BE developers. Few years ago when we had mixed UI and Backend in our projects each time FE developer had to do something in IBM Designer I saw such face.
I do not like such face :), one more reason why we forced our self to do what we did.

More details how FE developers works

I will try to describe workspace for typical FE developer on our project. First of all as I mentioned before they do not need IBM Designer, so they can use whatever they prefer. As all FE are located in GitHub after you import project you will get that (example from my Eclipse)
Once they commited anything to GitHub special mechanism would be triggered on our Jenkins build server and it will push all changes to Domino application.

Except many obvious benefits, any disadvantage?

Atm only 2 disadvantages:
  1. It takes ~60 seconds after you commit appear in Domino application (Jenkins server has to push all changed elements from GitHub to Domino)
  2. If you lost connection to internet - it's a problem (no access to GitHub)

Summary

If you do that you could get such benefits as: GitHub, freedom for FE, better mood for FE, new FE don't need to know anything about IBM Domino

Other articles in this series

  1. How we build our web applications based on Domino
  2. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
  3. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
  4. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
  5. Using Git+Jira is must and Jenkins server as builder server
  6. Auto-tests.

Tuesday, August 06, 2013

Split backend & frontend when build web applications

Is it really a problem to mix Backend (BE) and Frontend (FE)?

Yes, it is a problem, it's simply wrong. Some of you will argue with me and say:
  • I'm doing both: FE and BE and I'm 'pro' in both areas.
  • Our FE developers learned Domino platform in few weeks, so we have no problems, they feel comfortable with doing stuff with IBM Designer.
  • Our FE guys simply say what to change and we do that, they send us some template each time we need to change something.
  • We are doing internal website and our internal employee do not care about speed, modern UI etc.
I believe some of you are really perfect in both areas: FE and BE and you have enough time to improve skills in both areas just in time, so you don't follow practice that 10 years old. Honestly, at some point I thought I was good enough in both areas (FE and BE) to do modern stuff, however more I worked with web more I realized that my skills FE is not good enough to build really modern, fast and easy maintained web application. I spent years before I gave up and said to myself: I'm quite good in BE and I'm not bad in FE and that's actually enough, after that life became a bit more easy :).

Where is a problem?

Let's take an example when new FE developer joint to the team and he got tasks to change some UI.

1. Classic example (mail9.ntf, form memo)
Problems:
  • almost impossible to deal with that if you are not skilled in Domino;
  • have to know @Formula language to change <Compute Values> or hidden formulas;
  • have to know that fields might have some properties (classes, styles, id etc);
  • should ask for help each time they need to change UI;
2. xPages example (discussion9.ntf)
Problems:
  • more skill required about Domino, Java, SSJS and xPages especially;
  • controls will generate HTML automatically and it is a problem for our team as we aiming for 100% control;
  • as it's even more complicated, potentially FE will ask to involve BE developers each time they need to chagne UI.

If we split BE and FE then what?

Then life become much easier :), application will get better structure, things will go faster. How to do that? In our case we found template framework The Apache Velocity Project and tested it enough to make sure it would cover our needs. Then we started to move out UI from logic step by step, and yes, that process took months but once we did it we started to receive benefits immediately. Today we keep UI (HTML templates, CSS, JS) as files and our FE guys use GitHub. They are free to create new templates and organize their structure. They have huge freedom (compare to what they had 2 years ago) of how to do their job. There are really a lot of documentation + stackoverflow.com that can help. However Velcoity itself is quite intuitive and simple framework.

Wanna see 'Hello world' example?

In example bellow we pass 2 items to template (HashMap<String, String> map equal to Dim mylist List as String in LotusScript) then Velocity engine does rest of work and return result. In our project we simply pass all fields from document to template so FE guys have freedom and ability to work independently. It's really easy, proven by our FE guys.
HashMap<String, String> map = new HashMap<String, String>();
map.put("name", "Velocity");
map.put("author", "Joker");
velocity.applyVelocityTemplate("hello.world", map)
+++
## Velocity Template; id 'hello.world'
<html>
    <body>
       ## our hello world example
       Hello $name World!
       $author
    </body>
</html>
||
\/
<html>
    <body>
       Hello Velocity World!
       Joker
    </body>
</html>

Summary

Split and manage!

Other articles in this series

  1. How we build our web applications based on Domino
  2. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
  3. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
  4. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
  5. Using Git+Jira is must and Jenkins server as builder server
  6. Auto-tests.

Monday, August 05, 2013

How we build our web applications based on Domino

Intro

I've been working with Domino for many years (however very possible less than many of you :-), I started to work with version 5 at the beginning and in few months we migrated to 6.X). I worked in few companies with absolutely different projects and processes of development. There were lot of 'hell projects' (without any structure inside, just mess) and few really 'cool projects' where I could learn something. I always tried to bring the best of the old projects into new. Now I'd like to describe how we develop applications in Domino today. No revolution in our approach, however I believe that is quite good one. The only required thing in that process - you must have time and desire to change process.
Time to improve

Team and projects

We are small and quite typical team: few back-end and front-end developers, manager and no QA as we do not do mistakes :). Roll out each 2 weeks with new features. We are developing CMS that manage external websites of our company.

Few very important points about websites we are doing here:
  • website's pages have to load extremely fast (means 200-400 ms to load page);
  • clean HTML, all (!) tags, attributes, javascripts should be controlled;
  • easy roll out from development environment to production environment without any interrupt for users.

Process of development

Now let's look on most important points of development process. I'll describe each point more deeply in upcoming posts this week. Possibly I will add more points during that time, lets see.
  1. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
  2. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
  3. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
  4. Using Git+Jira is must and Jenkins server as builder server
  5. Auto-tests.
We've done almost all the list, some areas require some improvements however concept works and now it's only question of time. All front-end developers do not use IBM Designer in their daily job, back-end developers still use it (rarely), everything goes to GitHub, and then Jenkins take care about rest. That image illustrate what we have.
Domino development process overview
You are welcome to give your comments, idea, suggestions or something negative about that, everything can help and improve or process and wait for new posts soon :)

Other articles in this series

  1. How we build our web applications based on Domino
  2. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
  3. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
  4. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
  5. Using Git+Jira is must and Jenkins server as builder server
  6. Auto-tests.

Wednesday, February 27, 2013

Fighting for control of URL in Domino

We have really annoying problem for years with our URLs on all websites based on Domino. We use classic approach because we want control all tags + we like jQuery more etc. Our problem is about mandatory action [?open | ?opendocument] for pages with parameters, so if you want to add parameters to your page (i.e. param1=123) you have to add [?open | ?opendocument] just after your URL and only then you are allowed to add parameters.

Not really a huge problem, however we want to use new tracking code from google. That tracking has problems with [?open | ?opendocument]: it does not work correctly in case if first parameter is ?open. So now its blocker for us and we want to fix it for all another cases as well. We want to have possibility get rid off ?open | ?opendocument

Question in few words: "is it possible to get this URL working?"
http://www.e-conomic.com/accountingsoftware?parameter=123
instead of this
http://www.e-conomic.com/accountingsoftware?open&parameter=123

We've tried redirect, substitutions - no success ofc :(.

Next in my queue is DSAPI and I'm sure it can solve that problem (at least from documentation) however I'm not able to do that due to my experience with DSAPI and very very low information in the web.

I've played with different events in DSAPI: kFilterStartRequest, kFilterRawRequest and kFilterRewriteURL and tried to change incoming URL from request but did not success with it. Does any of you guys can give me tips :) what I do wrong or maybe there is another solution how to achieve that quest?

Monday, June 11, 2012

Default sort order

Guys, does anybody make it (Default sort order) working? I played to much with it without success, my Domino server still ignore sorting (as Swedish) for my application.



Tuesday, January 24, 2012

Error pages in Domino

First of all I'd like to briefly describe few another ways we can use to manage error pages in Domino.
Also notice, I will definitely update post few times more after all.

1. Lazy solution $$ReturnGeneralError and MessageString
It's the most fast and easy solutions, it require to create 1 design element (form) $$ReturnGeneralError, style it and add MessageString somewhere to explain what is wrong
Benefits:
- I see only 1 advantage compare to another approaches, its time to implement. Once you create $$ReturnGeneralError it starts to work. So few clicks and few minutes to manage output UI and you have solution.
Disadvantages:
- works inside of application only. So if you have 10 web applications, you need to manage error page in each of it. However you can setup inheritance and manage all error pages from 1 place etc.
- not very flexible on my opinion as we can operate with form-design elements only.

2. HTTP response headers
Simply create error page as design element or as document (does not matter) and create rule for your website in Domino Directory database. It does not require special knowledge and it is simple to use.

Benefits:
- easy to manage
- path to Error page (means we can use different database to keep error pages)
- we can use both: Design Elements and Documents as error page.
Disadvantages:
- require minor Administration of Domino skills.
it works in the same way as client side redirection. i.e. user will see “blink” of standard 404 page before loading custom 404 page.

3. "Whole server" solution
Use HTTPMultiErrorPage property in the Notes.ini file, for example HTTPMultiErrorPage=/error.html. Create rule on the Domino server for /error.html to be substituted corresponding page.
Benefits:
Disadvatages:

4. DSPAI as error handler for Domino
Most complicated approach but also most flexible from my point of view. There is quite low information about how to use it. It requires knowledge of Notes C Api as well. If you want to see how it looks, here is very good exampe (it helped me a lot) on loggin using DSAPI
Company I worked in use DSAPI for handling URL already, and now we are near to implement error handling using DSAPI as well (it already implemented it on development server, so soon I will show you real links etc).
DSAPI allows us to catch responses Domino generated for users and we can replace output in case if responce 400, 404 or any another.

I will post most important part of logic here, just to how overview how DSAPI works and what is gives to us. Notice I've changed code a bit, to show only most important part of it (I will copy comments from Paul's example just to make things faster) 

1. initiation of filter and register for response event
/*
* FilterInit() - Required filter entry point. Called upon
* filter startup, which occurs when HTTP server task is started.
*/
DLLEXPORT unsigned int FilterInit(FilterInitData* filterInitData) {
   filterInitData->eventFlags = kFilterResponse;
}
so what we did there, say to filter that we want to process response events
2. Link events we registered with functions
/*
* HttpFilterProc() - Required filter entry point. Dispatches the event notifications to the appropriate functions for handling the events.
*/
DLLEXPORT DWORD HttpFilterProc(FilterContext *pContext, DWORD dwEventType, void *pEventData)
{
 switch (dwEventType) {
  case kFilterResponse:
   return Response(pContext, pEventData);
 }

 return kFilterNotHandled;
}
Now we can process Responses to users
3. This is how we process Response to users
int Response(FilterContext* context, FilterResponseHeaders* eventData) {
 int responce = eventData->responseCode;

 if (responce==404) {
  if (Send404(context) == TRUE) {
   return kFilterHandledRequest;
  }
 }
 return kFilterNotHandled;
}
4. Finally code for Send404() how we replace content to users
While you read code you need to know that when we load filter we also initiate a table with KEY-HTML arrays. Yes we keep in memory Error HTML pages with keys we need. Our Keys - domain, we handle error pages on domain level, means for domain1.com - is 1 error page, for domain2.com is another error page. But you can do it in another way its up to you.
int Send404(FilterContext* context) {
 FilterResponseHeaders         response;
 unsigned int   errID = 0;
 char     szBuffer[DEFAULT_BUFFER_SIZE]={0};
 char     pszErrorPage[ERRRO_PAGE_SIZE]={0};
 FilterParsedRequestLine pRequestLine;
 unsigned int   pErrID;
 int      i;

 // As we are returning the entire page to the browser, we can 
 // set the response code to 404 (so the domino web log will show the error)
 response.responseCode=404;
 response.reasonText="Bad Request";
 
 // get domain name (its our key), could be done faster? right now its simple walk via 10-20 documents which is fine for now
 context->ServerSupport(context, kGetParsedRequest, &pRequestLine, NULL, NULL, &pErrID);
 for(i=0; i<errorPagesCount;i++) {
  if (strcmp(errorKey[i], pRequestLine.pHostName)==0) {
   strcpy(pszErrorPage, errorHTML[i]);
   i=errorPagesCount;
  }
 }

 if(strlen(pszErrorPage)<10) {
  sprintf(szBuffer, "Error page on %s is very small, something wrong", pRequestLine.pHostName);
  writeToLog(CRITICAL_MSG, szBuffer);
  return FALSE;
 }

 sprintf(szBuffer, "Content-Type: text/html; charset=UTF-8\n\nContent-length: %i\n\n", strlen(pszErrorPage));
 response.headerText = szBuffer;

 if (context->ServerSupport(context, kWriteResponseHeaders, &response, 0, 0, &errID) != TRUE) {
  sprintf(szBuffer, "Error sending redirect, code: %d", errID);
  writeToLog(CRITICAL_MSG, szBuffer);
  return FALSE;
 }
    if (context->WriteClient(context, pszErrorPage, (unsigned int) strlen(pszErrorPage), 0, &errID) != TRUE) {
  sprintf(szBuffer, "Error sending redirect, code: %d", errID);
  writeToLog(CRITICAL_MSG, szBuffer);
  return FALSE;
 }
 return TRUE;
}
OK guys, I've shared most complicated part of this DSAPI for error handling, rest you have complete yourself (homework :)), but feel free to ask my help here if you need.
benefits:
- most flexible approach (at least from those which I know)
- our logic control everything we want and we clearly see what is going on
disadvantages:
- most complicated ways from all I described
- require knowledge of Notes C API
- I did not try yet this solution with Linux servers (but I know it is possible to do)
- it may crash your Domino server in case if you did you filter wrong (memory leak etc)


5. xPage error handling is on way (on pause actually) and would be nice if somebody help me with that. I've read few articles in past from Per Henrik: XPages custom 404 and error page and Controlling the HTTP response status code in XPages, I think they can be very useful for those who doing in xPages. We will try this approach as we have ongoing huge projects based on xPage. 

Hey:
- ask to update/more details to article
- notify me about issues
- let me know if you know better way to handle errors

strongly recommended as it can help to many developers in future.

Tuesday, November 01, 2011

Solution for Lotus Domino to the trailing slash problem

2 months ago I've posted article about solution which can solve our problem in Domino with last trailing slash in URL. As I mentioned in my previous post, Domino does not care about url with/without last trailing slash, both way would work for Domino. Well, some of you maybe even find this feature useful because it gives less problems, however what if we talk about SEO (Search Engine Optimisation)?

There is actually article from google about to slash or not to slash, where they explain that they handle similar URL with and without last trailing slash as 2 different URL, they also propose ways how to solve this. If you ask what is bad here, answer is quite simple: just image that each URL has it's power/score for google. In case if we have only 1 possible URL for our page all score go to it, otherwise we will split them (high score will go to URL which is more often used by people in web).

Now about solution we implemented on our website. We used DSAPI to solve it as it does not do affect speed so much as different solutions. If you want to read more deeply about technical staff, you may want to read this article: solution to the trailing slash problem.

Yes it works like it should, no impact to page load's speed which was "very-very" important to us. And I heared google start to love us a bit more after that.

Related topics
DSAPI for Domino
Rewriting URL in Domino using DSAPI
Replacement for DSAPI in Java/XPages