Automatic Approval vs Multi-Destination Publishing

After the introduction of WebCenter Sites 11g8 version, Sites provides various improvements from creation of vanity url to proxy-assets to various customization hooks. One of the new customization hook introduced was - RealTime Publishing Customization Hooks which describes on how one can develop a custom solution for multi-destination/multi-target destination publishing. Although official guide provides full detail, I recommend to read this article once.

Thus, there is already an custom solution to:
1. Publish assets simultaneously to multiple destination. Furthermore, publishing to disaster recovery server can also be included.
2. Publish audit trails: Who published what and when
3. Minimize the effort of Contributors and Publishers

In one of my previous blog post - Automatic Approval of assets, I provided a glimpse on how to create a simple custom solution to automatically approve assets wherein administrators have control over setting up time when approval of content should occur which is similar as configuring scheduling publishing. With this approach:
1. Content is approved automatically to targeted destination
2. Admin has choice of selecting which content type (asset type) should be approved
3. Scheduling configuration just like scheduled publishing
4. If proper solution is developed, wherein administrators can easily disable/enable System Events, update asset types required for approval, destinations, scheduling, etc; automatic approval of assets can be helpful in various scenarios.

So, now which one to choose? Automatic approval or Multi-Target Publishing?
I have listed down few points which can help in deciding which mechanism will meet your requirements.

Features	Multi-Target Publishing	Automatic Approval
SCHEDULING	Although multi-publishing mechanism provides scheduling; it fails to configure different time settings for different destinations. This may be required sometimes when a piece of content is required to be shown at different times according to timezone and place; given that delivery servers are located at different geographical locations.	As automatic approval event itself is separate event, administrators can set automatic approval event for different target destination to different time as needed, furthermore, they can also set scheduling publishing as required.
EFFORT	This tasks will take only one step i.e. to build the solution and deploy it, that's it. Although it is one step process, if any regular changes required, it may require many changes within the solution; including restart of delivery servers which may not be desirable at all to customers.	Once the automatic approval element is ready which can intake assettypes and target destination as parameter, it is just matter of adding row in SystemEvents table. But if there is no way to update SystemEvents table via SiteExplorer (blocked due to security reasons), developers may need to build few more custom elements to add/update/delete approval events within SystemEvents table.
AUDITING	As already mentioned in developer's guide and the Oracle PDIT blog, one can also add methods within same implementation to include audit trails as required.	This mechanism can be very helpful in maintaining separate approval logs and publishing logs can be maintained.
NOTIFICATIONS	Publishing logs will contain full verbose output. But only those users with access to Admin UI can check the status and output. To access publish console from Contributor UI, will require another customization which is aptly described here - https://blogs.oracle.com/pdit-cas/entry/exposing_the_publish_console_in	Same customization of exposing publishing console can be helpful to know which assets were published. For approval messages, user can select the asset and check the status of each assets individually.
CONFIGURATION	After custom solution is ready, configuring multi-target publishing is very easy. It is the same process as one would configure publishing destination but with few more arguments.	Configuring automatic approval can be tedious task as admin has to manually add/update row within SystemEvents table. But if custom solution is developed for adding/updating rows, then it can become quite easy for administrators to maintain different approval events.
CAVEATS	One of the major problem with this functionality  is that it is required to restart all target servers as there would be changes made within AdvPub.xml file, but it is one time process. But there are other alternatives one can take like routing the traffic to other delivery server until other is restarted or content can be served from CDNS until all delivery server(s) is/are up and running. Another issue is even if any one of the target server is down, although assets were published to other running target server(s), assets are marked as NOT published for all the servers; whether they were running or not. Thus, assets will get published again next time  to all servers which may degrade performance depending on cache configurations.	One major problem is configuration i.e. adding or updating or deleting entry in SystemEvents table. But if custom solution is build for configuring automatic approval, then it is very easy for administrators. Another issue can be as it is more of custom solution built around WCS elements and not a separate customization hook, one has to make sure that during upgrade, custom elements are to be taken care of. Developers need to re-test the functionality again if OOTB approval element was changed.

So there you have it, all pros and cons of both mechanisms. There is no preferred or superior option to other. One has to weigh both the options according to cache configurations, architecture and of course, customer requirements. Furthermore, both mechanisms can be mixed and used as needed. For e.g. Multi Realtime publishing can be configured for delivery servers whereas test servers can be setup for automatic approval of content + scheduled publishing during no/less editorial activities.

Disclaimer: The code and/or the configurations posted are not official recommendations and should be used at sole's discretion with proper testing before deploying on live WebCenter Sites systems. Cheers!!

Deleting and regenerating cache

Caching is a very important aspect in Oracle WebCenter Sites and nearly all projects depend mostly on strong project design and optimal caching strategy (Note: Caching strategy also considers how your pages are coded). There is no use in going through various caching levels as they are aptly discussed in official document and in Oracle A-Team articles.

Before going through this post, I highly recommend to read about caching and different caches used within WebCenter Sites. This post is more helpful to those people who usually deal with resolving cache related issues.

What I want to highlight in this post is:
1. How to purge all cache?
2. How to re-generate all cache after all cache are purged?

One would ask why to delete all cache?
One of the important task while troubleshooting performance of a webpages is to calculate the number of cache objects generated, then by removing all the cache, hit a certain page(s) and find number of cache objects and dependencies generated for that/those page(s) using Support Tools or Admin UI. This task should generally carried out in Test environment or JSK. Thus, when it required to troubleshoot performance of webpages for full site(s), one can know by number of cache object generated for page(s) by flushing all the cache, make the changes within template or update cache settings and then re-generate all cache objects by hitting all the webpage urls.

How to purge all cache?
To purge all cache, one needs to know what kind of cache is enabled within your project i.e. incache (ehcache) or traditional cache (Only one of them is enabled). Incache is enabled by default within WebCenter Sites installation. Only old version of Oracle WebCenter Sites a.k.a FatWire 7.6 and before uses traditional cache but also has option to enable incache.

For deleting incache where WebCenter Sites is installed - Login to WebCenter Sites > Admin UI > Admin Tab > Expand System Tools Node > Cache Management > double click on Summary tab under CS Cache and Co-Res SS Cache separately. Select all checkboxs for flush cache and click "Flush Cache"
For deleting incache from Remote Satellite Server (RSS) - You can perform the same above step but using the following url:
http://[host]:[port]/[context]/Inventory?username=SatelliteServer&password=[password]

For deleting traditional cache, hit the following url:
http://[ip]:[port]/[context]/CacheServer?all=true&authusername=fwadmin&authpassword=[password]

One can use Support Tools to delete cache.

Furthermore, if you want to delete all incache forcefully, then you can find the location where cs-cache, ss-cache and linked-cache are stored from cs-cache.xml, ss-cache.xml and linked-cache.xml files respectively, which are located under [webapps]/cs/WEB-INF/classes folder and then delete everything under the respective folders. This task is to be performed after WebCenter Sites is shutdown.

How to re-generate all cache after all cache are purged?
Deleting cache is easy task but major problem is generating cache. A website can have hundreds to thousands of webpages and it is very arduous task to hit each webpage url one by one to generate cache. Also, WebCenter Sites does not provide any tool which can crawl through all the page urls. But there are various sites or tools available online which can do the task of crawling of all the webpages. One of the tools which I use and like is flem firefox add-on. It has ability to intake urls from clipboard memory and crawl through them as slideshow. It also has settings to set page load time interval and is very easy to use.

For using flex firefox add-on, basically one would need to first list down all webpage urls of site(s) and just copy & paste to flem add-on and run the slideshow.
If vanity url is configured for a site, then it is very easy to list all urls using the following query:
SELECT 'http://[ip]:[port]/[context]/' || webroot || '/' || webreferenceurl FROM WebReferences

Disclaimer: The code and/or the configurations posted are not official recommendations and should be used at sole's discretion with proper testing before deploying on live WebCenter Sites systems. Cheers!!

Developing website navigational section

Recently, one of our template was taking too much time to render left navigation section on a webpage, which displays all the pages under certain parent page upto level 3.

For reference, you can consider the following case:

Parent Page
- 1
  - 1.1
     - 1.1.1
     - 1.1.2
     - 1.1.3
  - 1.2
     - 1.2.1
  - 1.3
     - 1.3.1
     - 1.3.2
     - 1.3.3
- 2
  - 2.1
  - 2.2
     - 2.1.1
     - 2.1.2
     - 2.1.3
  - 2.3
- 3
  - 3.1
  - 3.2
     - 3.2.1
     - 3.2.2
     - 3.2.3
  - 3.3
     - 3.3.1
     - 3.3.2
and so on...

where, blue ones are level 1 page assets (placed under parent page), dark purple ones are level 2 page assets (obviously placed under level 1 page asset) and orange ones are level 3 page assets within FatWire/Oracle WebCenter Sites.

Similar configuration is present for other sites too as this project is configured for many countries (meaning many sites within OracleWCS and 2 or more dimensions per site)

I am discussing in detail on this topic so whomsoever reads this doesn't make the same mistake which our previous developer did.

Functionality required:
1. To render a particular page with left menu section listing page assets as shown above
2. The current page should be highlighted in left menu
3. On removing/adding page asset below Parent page asset within WCS at any level should reflect in left menu on next page load
4. Same functionality should be present for other sites (if there are multi-lingual sites)
5. And obviously dependencies and caching should be taken care automatically (no publishing of assets or flushing dependencies/caches)

Let me first discuss on how usually developers code navigational elements. Considering the above example i.e. to render left menu; one can think of a logic like: to call certain Template/SiteEntry (say LeftNav, assettype-Page, usage: Element is used within HTML page, cached) by adding following parameters to cache criteria: c, cid, site, locale and other common parameters (if required); which will be called from Layout template.
Now to build left navigation from top-to-bottom, one needs to know parent page id. So there are 2 ways, either you add parent page id to MAP tab of Template/CSElement for every site or load the current page's siteplan and find the parent page by traversing loaded siteplan (One can argue that same can be achieved if there is basic asset which stores only key-value pairs and parent page's asset id can be stored per site but again there is risk of duplicity of asset name, not to mention it is always prone to human-errors).

First case (getting value from MAP) is generic. For latter case, one can do the following:
<asset:load> - Load current asset from the param - cid
<asset:getsitenode> - Get currently loaded asset's node id in SitePlan Tree
<siteplan:load> - Load siteplan by passing the node id obtained above
<siteplan:notepath> - Get list of ancestors
Last entry in the IList will fetch the parent page asset's asset id and node id

Once you have parent page, start by loading the siteplan from this parent page id or node id and build the left navigation:
<asset:load> - Load current parent page asset
<asset:getsidenode> - Get node id of parent page
<siteplan:load> - Load the site plan
<siteplan:children> - Get first level of children
and so on... repeating same process till required level is reached.

Drawbacks of using above logic:
1. If the Template/SiteEntry for LeftNav is cached, there would be many left menu pagelets generated resulting in non-desirable unnecessary over-caching of a page section. For e.g. if you have 50 page assets under parent page asset, then logically you will have 50 cached enteries for left menu per site. And suppose, there are 10 sites configured (multi-lingual), then total number of cached pagelets = 500. Futhermore, consider this -> Even if there are 2 dimensions configured per site within your WebCenter Sites/FatWire project, then there would be 1000 cached pagelets only for left navigation/menu!! At last, one would think that if there are so many cached enteries, developer's end up leaving LeftNav Template/SiteEntry uncached.

2. If there is only site within project, then retrieving & handling value from MAP or using basic asset should not be a problem, but if there are more than 10 sites present in your project, then it may or may not problem; again depends on project design. If you are using basic asset to store key-value, one needs to always make sure if correct page asset id is added as value or not; which is always prone to human-errors. Hence, one can't escape from loading the current page siteplan and then finding the parent page's asset id or node id. There might be some other simple way too.

3. Last but not the least is the issue with loading siteplan for every page asset as you loop through IList from top-to-bottom. Hence, if considered 50 page assets per site under Parent Page, then 51 siteplan loads!! (including parent page) Furthermore, if 10 sites are present within a project, 510 siteplan loads!!  Also, one doesn't know when to stop loading children pages until it is cleared upto which level page assets are to be searched for. Some page asset may or may not have child page assets but still siteplan:children tag is loaded while looping from top-to-bottom (also have to add extra if-else condition for child pages existence). This unnecessary loads using siteplan:load tag will definitely affect page responsiveness for every new page request. Developers should be really good in developing such elements and the depth needs to be fixed before implementation too. Additionally, one not only needs page id, but also require page attributes or association and generate url to show in left navigation. Also, you will add some condition to highlight the current page in left navigation which may or may not be complex (depends on HTML prototype). Considering all this, if you check the load time using <time:set> and <time:get> tags, that would be high compared to other elements.

In your project, if this is how navigation/menu is built; then this is really a bad implementation on developer's part and requires inspection.

So how to avoid this?
Coding such scenario is not much of challenge until you know the proper way to implement it. If you check FirstSiteII, similar case is present: Loading product categories in left navigation on product page but it was using searchstate tags whereas in this case, siteplan tags are used.
I am listing few suggestions which can be implemented to increase page performance and reduce unnecessary generation of pagelets.

1. Pagelet caching: Considering that every page request goes through layout (or wrapper+Layout) template and each layout template calls various parts of your webpages - header, footer, navigation and body. For our case, layout template will call an extra pagelet element - LeftNav. Create LeftNav template (same configuration as before but make it as uncached). Create another template - LeftNavChildren (or whatever name) which will be called from LeftNav template; having same configuration (cached, for page asset and usage as HTML pagelet, include same params). When a page having left navigation is requested,  LeftNav (uncached) will just calculate the parent page's asset id or node id from the current page cid's value and call LeftNavChildren template by passing this calculated parent page's asset id. As we will always get same parent's page asset id from LeftNav template for a site for every page asset in left navigation, there would be only one cached LeftNavChildren per site per locale. Hence, considering same scenario as 1st point in drawbacks section, we will now have only 1 pagelet * 2 dimensions * 10 sites = 20 cached pagelets!! (Awesome, isn't it??). Hence, in one word, try to cache on parent's page cid not on every page asset in left navigation.

2. Loading parent page: There is not much to say for this case as its very obvious - if more sites are present within project, then its better to use tags to get parent page id or else use MAP tab or some other way.

3. Loading child pages: Use siteplan:listpages tag rather than loading child page assets using siteplan:children tag. This tag has ability to search all the child page assets in one go; given that you provide parent page node id and the level of depth upto which search is required. Hence, no messy code. Following few lines will completely fetch siteplan from top-to-bottom for left navigation.
<asset:load> - Load current parent page asset
<asset:getsidenode> - Get node id of parent page
<siteplan:load> - Load the site plan
<siteplan:listpages> - Get list of child page assets of this parent page upto whatever level of depth required.

Resulting IList contains list of child page assets containing important columns: node level (level 1, 2, 3...), id (page asset id) and hasChildren (page asset has children or not). Hence, if coded efficiently, LeftNav execution time can be reduced drastically. Here is a sample element which can load any siteplan by providing page id and level of depth to search & shows tree like navigation.

4. Highlighting current page: As we are not passing current page id, one may ask how to highlight current page in left navigation? Well, its very simple. By Comparing URL or part of URL generated in left navigation with the current page URL (as URL in left navigation for the current page will be same as the URL in the browser, not to mention that URLs will be unique for a site). Hence, a little knowledge on JavaScript/jQuery can do the trick.

There might be few more other tweaks which I may have missed but these are the essentials to know as every project has requirement to develop navigational section like footer, left navigation, breadcrumb, top navigation or sitemap. I really hope this make things clear on developing navigational part of a website and helps many FatWire/Oracle WCS developers.

Furthermore, this article is a must read for any WCS developer.

Disclaimer: Any sample code on this blog is not officially recommended, use at your own risk.

Adding constraint on existing asset list

In one of my project, I faced dire situation where in I had to add search condition on existing attribute (say A1) of type asset on already searched assetlist.
As we were using lucene search, to search against this attribute: A1, default way will be to add one filedcopier filter, so that it can copy id from custom attribute, A1 to some other custom attribute & then should add the condition for it in search element. Basically, I had to show results by previous conditions + search on A1's attribute value (if exists). For e.g. Previously, if I got 500 results on searching, I should now get results by searching against previous conditions + search results from searching against custom attribute, A1; basically adding one more constraint on searching assets.

Sounds straightforward and easy, right?? But no, clients will be clients. Issue with adding filter is that you have to edit and save all the assets so that filter is executed. But by doing so, updateddate field value's changes and which client did not wanted that to happen at all for auditing purposes. Damn!!

So, basically now I had only one option i.e. to add one if condition while iterating through previous results. Meaning i would have to load each asset, get its attribute: A1 value and check if A1 attribute's value exists for that assets or not & matches with desired value or not. Obviously desired result will be generated but it would be worst non-desirable algorithm.

Hence, I started looking for Tags and APIs to avoid the such bad scenario. And to my luck, there are 2 tags which does what was required for my case - <assetset:setsearchedassets> and <assetset:setlistedassets>

<assetset:setlistedassets> tag can build assetset by consuming assetlist (as input list) but should have columns: assetid and assettype. Even if you don't have this columns within your IList, you can build your IList using <listobject> tags with columns: assetid and assettype

Once you have  assetset, it becomes straightforward to add constraint by creating searchstate and passing this constraint to <assetset:setsearchedassets> tag

Overall code would become like following (assetlist is input list with columns: assetid and assettype):

<assetset:setlistedassets name='thisassetset' assets='assetlist' deptype='exists'/>
<searchstate:create name="ss" op="or"/>
<listobject:create name="lo" columns="value" />
<listobject:addrow name="lo">
    <listobject:argument name="value" value="[some value]"/>
</listobject:addrow>
<listobject:addrow name="lo">
    <listobject:argument name="value" value="[some other value]"/>
</listobject:addrow>
<listobject:tolist name="lo" listvarname="matchlist"/>
<searchstate:addstandardconstraint name="ss" attribute="A1" list="matchlist" immediateonly="true" typename="[Attribute type name]" bucket="filterBucket"/>
<assetset:setsearchedassets name="thisassetset" constraint="ss" assettypes="Content_C" site="
    <%= siteId%>"/>
<assetset:getassetlist name="thisassetset" listvarname="assetlist"/>

Voila!! Now, this output list - assetlist returned is desired filtered list with criteria passed.

Disclaimer: Any sample code on this blog is not officially recommended, use at your own risk.

Implementing pagination on search results

I have been searching an easy way to implement pagination which requires no jQuery/JavaScript or java class implementation. And to my surprise, I found one element - Search.jsp which was present with product installation itself!! It's located at OpenMarket/Xcelerate/Search/Search.jsp which gave me basic idea on how pagination can be implemented with ease within WCS or FatWire.

Requirements:

1. Your search should be implemented via some search engine like lucene or endeca as they have ability to get results by setting starting index and max results. I have used by default lucene search which is installed out-of-the-box with JSK.

2. Knowledge on implementing forms within WCS. Check details here.

3. Knowledge on implementing lucene search. Check details here.

Procedure:

1. Basic idea is to generate page url with some params which lets your application know what results to show (in other words its just passing different params to your page asset templates). Generate URL same as which you generated for search button - Use <render:gettemplateurl> - Either you get vanity url if configured else you will get url by your assembler or generic long url

2. Append all the input params as query strings - Just append all the input params which were passed to search the current page. You might want to consider using packedargs if many parameters are required to be passed. Use tag: <render:unpackarg> and <render:packargs>

3. Update lucene search logic to search assets by setting start and end index - Simply add the following lines:

int startIndex = (null == pageNum || pageNum.length() == 0 ) ? 0 : (Integer.parseInt( pageNum )-1) * 10;

int endIndex = (null == pageNum || pageNum.length() == 0 ) ? 10 : (Integer.parseInt( pageNum ) * 10);

finalQuery.setStartIndex( startIndex );

finalQuery.setMaxResults( 10 );

where, pageNum = page number like 1,2... if null then just pass 0 as starting index and 10 as endIndex. I have considered that I would be showing 10 results per page. By adding this condition, lucene fetches only those results which are asked for.

4. Atlast, a simple logic to generate pagination

if(numOfArticles>10){%>   
    <ul id="pagination" style="padding-right: 50%;padding-left: 50%;display:inline;">   
    <% int pageCount = 0;   
     for(int p=0; p<numOfArticles; ) {   
        pageCount++;   
        String url = ics.GetVar("actURL")+"?keyword="+keyvalue+"&n="+pageCount;    
       if(Utilities.goodString(pageNum) && Integer.valueOf(pageNum) == pageCount ||       !Utilities.goodString(pageNum) && pageCount==1){   
        %><li style="margin: 0 10px;display:inline;"><a style="font-size: large;"><%= pageCount       %></a></li><%   
        } else {   
           %><li style="margin: 0 10px;display:inline;"><a style="font-size: larger;text-      decoration:underline;" href="<%= url %>"><%= pageCount %></a></li><%   
        }   
        p = p + 10;%>   
     <%} %>   
     </ul>   
<%} 

where, numOfArticles = total number of assets found on search

actURL = search results page url

pageNum = pageNumber like 1,2... which will be passed to URL for all the pagination links

rest of code is simply beautification.

If you have followed instructions mentioned in the posts for Satellite form and lucene, then building pagination should not be tough. 

I have developed and shared one sample JSP element which works with latest 11g JSK + Vanity URL configured + patch2 applied. Make changes in template according to your installation and configuration.

Add the param - "n" to cache-criteria to your previous SearchLayout.jsp and the replace code with the new pagination based search results - SearchLayoutPagination.jsp. Your page should look like following:

Note: There are differences in rendering forms for latest 11g and old versions, do check out how to work with them before trying this example.  

Disclaimer: Any sample code on this blog is not officially recommended, use at your own risk.

Engage: Flushing visitor's data

If you are using Engage, then is one caveat which needs to be taken care of:

"Data in visitor's tables can increase exponentially (obviously depends on how many times your site is visited) which can affect performance"

FatWire/Oracle WCS has no way to know which data is to be deleted and on what basis so basically you have to flush data from tables either manually or by programming. 

Hence, following sample code snippet can help to flush data:

But wait, does admin has to run this code again and again? No. WebCenter Sites has a very important table: SystemEvents where one can add SiteEntry+CSElement or Template and set a time to call this element. Simple, isn't it? So just create an entry in SystemEvents table and some logs, if required.

Tag info: http://docs.oracle.com/cd/E29542_01/apirefs.1111/e39371/JSP/vdm-flushinactive.html
http://docs.oracle.com/cd/E29542_01/apirefs.1111/e39371/JSP/vdm-mergeinactive.html

Disclaimer: Any sample code on this blog is not officially recommended, use at your own risk.