Welcome!

Java IoT Authors: Derek Weeks, Liz McMillan, Automic Blog, Karthick Viswanathan, Stackify Blog

Related Topics: Java IoT

Java IoT: Article

Managing a Standardized Build Process Outside of the Eclipse IDE

Point-and-click solutions won't cut it

Building objects in the Eclipse IDE is simple - it's a point-and-click solution. However, as applications built on the Eclipse platform mature the need for building outside of the IDE increases. This need can be driven by the development team that is striving to perform agile development techniques where builds are executed based on a file "check-in" action into an SCM tool. The need can also be driven by IT governance where a scheduled and audited production build is required. Moving from builds managed inside of the Eclipse platform to builds managed outside of the Eclipse platform can be a big task in itself. Don't hesitate to make this jump. It's a jump that you'll find you can't do without. The sooner you get out of your point-and-click build process, the sooner your application will begin to mature.

Defining the build process should not be taken lightly. Auditability and traceability of built objects are becoming increasingly important due to IT compliance mandates. This means that your builds must become more traceable than a point-and-click process. Don't make the mistake of addressing the issue of building outside of the Eclipse IDE long after the application has grown to an unmanageable size. Delaying the inevitable only results in a poorly managed, unplanned, ad hoc build process that isn't sustainable, can't meet IT compliance, and involves expensive hidden costs in maintenance and fixes.

There are three ways of addressing the build process outside the Eclipse IDE. The most common method is to manually develop and maintain Ant/XML scripts. These scripts use Ant Tasks from the Apache Foundation to act as a wrapper to the Java compiler. The second method is to write scripts that call Eclipse in what's called a "headless" mode. A script that executes the build in headless mode acts in the same way as the point-and-click process inside the Eclipse IDE, but does the build from a script. And finally, the preferred method is to use a commercial build tool that can automate the creation of the scripts. Commercial tools that minimize the use of manual scripts establish a more repeatable and traceable build workflow, the ultimate goal of any solid development process.

If you don't have the luxury of a commercial build tool that can create a solid reusable build framework, you must create a manual build process that's as standardized as possible. A "manual" build process refers to any scripted build process that has to be maintained manually. Even if you execute your manual scripts through a job scheduling build management tool, your builds are still manual because you must manually maintain the build logic contained in the scripts.

When writing the manual build process, your choices become writing Ant/XML scripts to perform the build or to use the Eclipse headless mode option. Establishing a repeatable build that can be traced requires that you concisely define how the build executes and with what source code. The use of the headless mode removes that level of control. It provides a little more functionality than using the point-and-click process inside Eclipse. When running a headless mode script, you're still relying on the Eclipse IDE to control the build. For this reason, defining a build process using Ant/XML is recommended if a commercial tool isn't available.

If you review any Ant/XML script, it may appear that the process of converting Java source into Java jars is complicated. This isn't necessarily true. Ant/XML files execute in a serial fashion, top to bottom. Everything must be precisely coded in a particular order. That's why XML build scripts can be very large and difficult to debug. There are some suggested standards on writing XML scripts, but they're not always followed. At a minimum, XML build scripts should follow a basic flow with pre-processing and post-processing steps that are consistent for every XML script created, planned, or unplanned.

Pre-Processing Steps
Pre-processing steps are used to establish the environment in which the subsequent task will execute. The point of setting up these pre-processing steps is to get the source code and variables organized and do overall housekeeping before compilation starts. A common mistake is to mix together these housekeeping steps for each call to the Java compiler or before calling an Ant Task. By organizing the pre-processing steps at the beginning of the XML scripts, the build process becomes clearer and easier to follow. It also reduces redundancy and results in an improved, more efficient process. These are the recommended pre-processing steps:

ClassPath
The CLASSPATH identifies which Java Classes are going to be used to resolve inter-class dependencies. The Java compiler will search the CLASSPATH in a first-found method and use a file as soon as it's been located. The CLASSPATH can include Jar and Zip files and directories.

When setting the CLASSPATH it's important to make sure of two things. First, the CLASSPATH should only be defined once in your process. Second, only the jar files and class directories that are used should be referenced in the CLASSPATH. Don't reference other unused jars since then the Java compiler will do more work than needed, slowing down your build substantially. And specifying only the used jars will make for quick dependency identification. There's nothing worse than attempting to trace jar file dependencies only to find that a large number of jar files aren't needed. This can add substantial time to debugging your builds.

The use of wild cards is always a topic of debate. Wild cards can eliminate typing in your script, but in the end may cause your build process to include more objects than needed. List each jar file in the CLASSPATH explicitly to prevent any vagueness. This is particularly critical when exposing your build details for IT compliance mandates. Wild cards aren't traceable.

Setting the CLASSPATH should be the first task in the Ant XML script.

Copying and Renaming Files
There are cases in which source code, jar and property files have to be copied from one location to another for the compiler to find the file or put it in the archive correctly. Do yourself a favor and minimize the use of copying and renaming of files. It makes it extremely difficult to trace the archive contents back to the original source. Copying files around also creates a more "magical" build process. IT compliance mandates want a clear view into your process. No "magic" is needed.

Instead of copying or renaming the files, put the files in the correct location from the start. Don't use your Ant/XML script to clean up a mistake in file organization. This may involve updating your project directory structure and making your Source Code Management tool more efficient. As an alternative to copying and renaming files, the use of the Ant Task "Zip" and its attributes, such as "dir" and "prefix" can handle getting source from one location and putting it in the archive at a different location.

This sample XML code from the Apache Ant Manual demonstrates using the Ant Task "Zip" to take one source location (htdocs/manual) and put it in another location (docs/user-guide):

<zip destfile="${dist}/manual.zip">
     <zipfileset dir="htdocs/manual" prefix="docs/user-guide"/>
     <zipgroupfileset dir="." includes="examples*.zip"/>
</zip>

Compile Step
The compile step is, of course, the heart of your process. It will become the largest section of XML script. The important point to remember when defining this portion of your script is the management of dependencies.

Dependency References
With Ant, you can explicitly define the dependencies between tasks. For example, the JAR task can be dependent on the JAVAC task. Ant will also let multiple task dependencies be established. Don't be seduced by this seemingly convenient Ant Task. While it seems useful, it can be burdensome. When tracing the order of executing the various Ant tasks in the Ant/XML script, it's much easier to follow a dependency chain that has only one task dependency instead of multiple ones. For example:

Scenario 1
JAR Task depends on JAVAC Task
JAVAC Task depends on the COPY Task
COPY Task depends on the INITIALIZATION Task
Versus

Scenario 2
JAR Task depends on the JAVAC Task and COPY Task
JAVAC Task depends on the COPY Task and INITIALIZATION Task
COPY Task depends on the INITIALIZATION Task

As you can see in Scenario 2, there are redundant Ant task dependencies. For example, the COPY Task is redundant on the JAR Task. This redundant use of COPY Tasks isn't needed since it's already referenced higher up in the task dependency hierarchy being the JAVAC Task.

There will be cases when you want to have multiple task dependencies as in the creation of a war file. In this case, multiple task dependencies may be needed to ensure that all of the jars are created before the war. But each jar should have just one task dependency, that being the JAVAC task.

Identifying Source Code
Finding your source can seem like an easy item at first, but when applications get bigger there's a greater chance that wrong or obsolete code is included. Using wildcards in the Ant/XML script is an easy way to minimize the need for typing, but for the wildcards to be effective, the source files have to be efficiently organized in a proper Java package directory structure.

The best way to manage source is to define an efficient package directory structure. So you must move beyond your unique needs and address the package names at a more global level in your organization. It's best to make sure that a corporate Java package structure is agreed on and used. As part of the Java package structure it's best to keep the package names simple. Really long package names can cause problems with the file limits on the Windows operating system. Java compiles on Windows have been known to stop working when the 254-character limit is exceeded. To make this problem even peskier, the script may work on one user's machine but break on another's. This is due do the build directory root name being added to the package names. For instance, one person may do the build in c:\mybuilds but another may build in d:\onlinedata\j2ee\development\code. The difference in the directory name can make or break the build by pushing the 254-character limit.

Another item that defines the location of source code is the use of the excludes attribute of the JAVAC Ant Task. It's best to remove the older obsolete code from the Source Code Management tool and from the file system instead of using the exclude attribute. Most SCM tools provide for renaming or removing an item without loosing all of the history. SCM tools also allow for comments that create a level of traceability on why a piece of code is no longer required. Having this information in the SCM tool makes for easier access versus the information being hidden in a comment in the Ant XML.

JAVAC -sourcepath
The native command line Java compiler (javac.exe) has an interesting flag called -sourcepath that provides a directory concatenation to find the source code. It works on a first-found basis. So once the source has been located, the directory browse stops. There are two advantages to using this parameter. First, all of the code doesn't have to be found in the current build directory. That is, source code can be found in multiple locations. Thus, the build process only has to check out the changed code and find the remaining code from a previous full checkout. This will speed the overall build process by minimizing the files to be checked out. Second, if the JAVAC command is given a Java file as a parameter it will then check using the -sourcepath parameter for additional source that's referenced by the original Java file and compile it too. This process will allow just the changed source to be passed as the parameters to JAVAC and JAVAC will figure out all the remaining dependencies for you.


More Stories By Steve Taylor

Steve Taylor is an experienced senior developer, bringing 17 years of expertise with client/server and mainframe application development and system integration. Prior to founding Catalyst Systems Corporation, he served as a lead technical consultant responsible for the successful implementation of applications into the production environment. Steve received his BS in computer science/mathematics from the University of Illinois-CU.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


@ThingsExpo Stories
WebRTC is great technology to build your own communication tools. It will be even more exciting experience it with advanced devices, such as a 360 Camera, 360 microphone, and a depth sensor camera. In his session at @ThingsExpo, Masashi Ganeko, a manager at INFOCOM Corporation, will introduce two experimental projects from his team and what they learned from them. "Shotoku Tamago" uses the robot audition software HARK to track speakers in 360 video of a remote party. "Virtual Teleport" uses a...
SYS-CON Events announced today that Datera will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Datera offers a radically new approach to data management, where innovative software makes data infrastructure invisible, elastic and able to perform at the highest level. It eliminates hardware lock-in and gives IT organizations the choice to source x86 server nodes, with business model option...
When shopping for a new data processing platform for IoT solutions, many development teams want to be able to test-drive options before making a choice. Yet when evaluating an IoT solution, it’s simply not feasible to do so at scale with physical devices. Building a sensor simulator is the next best choice; however, generating a realistic simulation at very high TPS with ease of configurability is a formidable challenge. When dealing with multiple application or transport protocols, you would be...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, Cloud Expo and @ThingsExpo are two of the most important technology events of the year. Since its launch over eight years ago, Cloud Expo and @ThingsExpo have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, I provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading the...
In his session at 21st Cloud Expo, Carl J. Levine, Senior Technical Evangelist for NS1, will objectively discuss how DNS is used to solve Digital Transformation challenges in large SaaS applications, CDNs, AdTech platforms, and other demanding use cases. Carl J. Levine is the Senior Technical Evangelist for NS1. A veteran of the Internet Infrastructure space, he has over a decade of experience with startups, networking protocols and Internet infrastructure, combined with the unique ability to it...
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business - from apparel to energy - is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
SYS-CON Events announced today that Elastifile will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Elastifile Cloud File System (ECFS) is software-defined data infrastructure designed for seamless and efficient management of dynamic workloads across heterogeneous environments. Elastifile provides the architecture needed to optimize your hybrid cloud environment, by facilitating efficient...
SYS-CON Events announced today that GrapeUp, the leading provider of rapid product development at the speed of business, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company, specialized in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market acr...
"We provide IoT solutions. We provide the most compatible solutions for many applications. Our solutions are industry agnostic and also protocol agnostic," explained Richard Han, Head of Sales and Marketing and Engineering at Systena America, in this SYS-CON.tv interview at @ThingsExpo, held June 6-8, 2017, at the Javits Center in New York City, NY.
SYS-CON Events announced today that Golden Gate University will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Since 1901, non-profit Golden Gate University (GGU) has been helping adults achieve their professional goals by providing high quality, practice-based undergraduate and graduate educational programs in law, taxation, business and related professions. Many of its courses are taug...
Recently, IoT seems emerging as a solution vehicle for data analytics on real-world scenarios from setting a room temperature setting to predicting a component failure of an aircraft. Compared with developing an application or deploying a cloud service, is an IoT solution unique? If so, how? How does a typical IoT solution architecture consist? And what are the essential components and how are they relevant to each other? How does the security play out? What are the best practices in formulating...
SYS-CON Events announced today that Cloud Academy named "Bronze Sponsor" of 21st International Cloud Expo which will take place October 31 - November 2, 2017 at the Santa Clara Convention Center in Santa Clara, CA. Cloud Academy is the industry’s most innovative, vendor-neutral cloud technology training platform. Cloud Academy provides continuous learning solutions for individuals and enterprise teams for Amazon Web Services, Microsoft Azure, Google Cloud Platform, and the most popular cloud com...
In his session at @ThingsExpo, Arvind Radhakrishnen discussed how IoT offers new business models in banking and financial services organizations with the capability to revolutionize products, payments, channels, business processes and asset management built on strong architectural foundation. The following topics were covered: How IoT stands to impact various business parameters including customer experience, cost and risk management within BFS organizations.
DX World EXPO, LLC., a Lighthouse Point, Florida-based startup trade show producer and the creator of "DXWorldEXPO® - Digital Transformation Conference & Expo" has announced its executive management team. The team is headed by Levent Selamoglu, who has been named CEO. "Now is the time for a truly global DX event, to bring together the leading minds from the technology world in a conversation about Digital Transformation," he said in making the announcement.
SYS-CON Events announced today that DXWorldExpo has been named “Global Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Digital Transformation is the key issue driving the global enterprise IT business. Digital Transformation is most prominent among Global 2000 enterprises and government institutions.
In his opening keynote at 20th Cloud Expo, Michael Maximilien, Research Scientist, Architect, and Engineer at IBM, discussed the full potential of the cloud and social data requires artificial intelligence. By mixing Cloud Foundry and the rich set of Watson services, IBM's Bluemix is the best cloud operating system for enterprises today, providing rapid development and deployment of applications that can take advantage of the rich catalog of Watson services to help drive insights from the vast t...
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Me...
From 2013, NTT Communications has been providing cPaaS service, SkyWay. Its customer’s expectations for leveraging WebRTC technology are not only typical real-time communication use cases such as Web conference, remote education, but also IoT use cases such as remote camera monitoring, smart-glass, and robotic. Because of this, NTT Communications has numerous IoT business use-cases that its customers are developing on top of PaaS. WebRTC will lead IoT businesses to be more innovative and address...
In his session at @ThingsExpo, Sudarshan Krishnamurthi, a Senior Manager, Business Strategy, at Cisco Systems, discussed how IT and operational technology (OT) work together, as opposed to being in separate siloes as once was traditional. Attendees learned how to fully leverage the power of IoT in their organization by bringing the two sides together and bridging the communication gap. He also looked at what good leadership must entail in order to accomplish this, and how IT managers can be the ...