Progressive Decoupling With Drupal 8 and ReactJS

Every new project is a chance to discover or revisit a wide range of technologies and methods that might help us bring to life what our clients and design team have been visualizing for weeks. But even as my team posts links to new libraries, frameworks, and processes in our Slack channel, I have to be careful that we don’t simply adopt something because it’s the newest and shiniest offering.

In this post, I’ll walk through the background of a project that we recently completed, explain how we arrived at the decision to implement a progressively decoupled portion of the site, and list some things that we learned along the way that will help us avoid some of the stumbles and lost time on future projects.

Background

Back in January, my team began scoping a new site for the University of Pennsylvania’s Office of Undergraduate Admissions. One of the functional requirements was that visitors (primarily prospective students) would be able to choose a variety of topics that they were interested in, and as they moved throughout the site, various student stories or “opportunities” — small bits of text that would appear in the sidebar, detailing things that students could do outside the classroom related to the selected interests — would need to “bubble up” to the top of lists. At any point, prospective students needed the ability to add or remove interests, and the page had to immediately update to show different stories and opportunities. As the visitor moved to a new page, those interests needed to follow them.

As we discussed various ways of implementing this functionality, I recommended that we use ReactJS components for the various widgets that needed to respond to user input, as well as the story and opportunity cards that would display content to the visitor. To maintain state between page loads, we would write a small custom module to store a visitor’s interests and visited pages in the database, tied to a randomly-generated UUID, which we would store in a cookie. Custom endpoints in that module would handle changes to the database (adding/removing interests, marking a story or opportunity as “viewed”), while Drupal’s own RESTful Web Services API would supply a few views to front the interest, student story, and opportunity content to the React app.

Ultimately five developers, myself included, contributed code to the React portion of the site. All five of us have strong vanilla JavaScript skills and found that we were able to hand off various aspects of the components to one another with ease and a minimum of “spooling up.” After some extensive performance refactoring (explained below), I felt that we finally had a solid product that could also serve as a boilerplate for future decoupling projects.

Lessons Learned

This portion of the project was certainly a learning experience for everyone who worked on it. Thankfully, we were able to identify a number of lessons that we can take forward to the next React project, both in terms of “how do I write this code” and “how do I architect these components.”

Here, in no particular order, are some of those lessons:

Small-scale performance might not scale up well.

As my team scoped out the functionality, I recognized that components would be placed on the page, usually in isolation from one another, and therefore made the decision that each component should be completely capable of fetching the information it needed to display properly from the Drupal backend without relying on any other component. This worked great when there were only five or six student stories loaded: the component that rendered stories was super fast to retrieve the data every time it was placed on the page; and even when all six stories were placed on a “list” page, they all loaded quickly. What I didn’t anticipate was how many stories would ultimately be created by the client’s content team. Some pages ended up with 30 or more stories and took upwards of 90 seconds to load on a desktop browser, and would simply crash on mobile devices.

Decide early on if you need global state. (Spoiler alert: You probably need it.)

As we initially scoped out the functionality, we agreed that we probably wouldn’t need a global state manager like Redux (or a similar alternative) to manage global state. As the application grew, however, it became increasingly obvious that we should have implemented something because, as demonstrated in the previous lesson learned, I discovered I was constantly fetching the same data. Furthermore, I realized that changes to one component (selecting a new interest, for example) would need to update any number of additional student story or opportunity components on the page.

Ultimately, as I refactored these components, I decided against trying to insert global state management that late in the process and instead opted to use PubSubJS to implement a notification system that passed new state to any component on the page that was listening for it. This worked because there was only one endpoint that returned variable data, and it was only ever called when the visitor changed their interest selections.

Don’t go crazy with API calls.

Two performance problems surfaced once I combined the various React components: Individual components would make their API calls (sometimes to three different endpoints) even if it meant fetching the same data from the same endpoint multiple times; and individual components would make those API calls in the componentWillMount() method, which, I didn’t realize at the time, meant every time a component was placed on the page, even if it had been on the page previously. In the worst case, a list page that showed all student stories and opportunities related to a visitor’s selected interests would display every interest, and every story and opportunity for each interest, if no interests were selected; and a single click (for example, unselecting the last interest) could kick off hundreds of calls to the Drupal backend, most of which were asking for the exact same small bit of JSON. Another factor that didn’t help my performance problems related to the API itself: I crafted my endpoints such that accessing /student-stories would return a list of all student stories, while accessing /student-stories/123 would return just the JSON for the story with the node ID of 123. Many times, in the spirit of “don’t ask for more than you need,” one component would fetch all available stories, while another would fetch just a single story.

My solution here was, as outlined above, to fetch static data once per page load, and share that initial prop out to all the components; but also to eliminate the single-node-ID endpoint and instead locate the appropriate record from the static list of all content. So, for example, rather than fetch all stories, then separately fetch story 123 in another call, I would simply look back at the prop containing all the stories and pull out story 123 when I needed it. This greatly reduced the number of calls being made to the server, and now each page load only makes those initial three API calls, rather than several hundred duplicate calls.

Behavioral tests are a MUST.

Once we integrated the components, we started fielding bug reports from the client, which was to be expected. However, we kept creating regressions as we fixed bugs. Architectural decisions that seemed pretty small started coming back to haunt us as we uncovered interactions we weren’t expecting between components. While we have implemented visual regression testing on all of our new projects moving forward, it wasn’t nearly enough to test the behavioral interactions between components.

This is the only “lesson learned” that we haven’t closed out yet. One of our “summer research projects” on the dev team is to investigate ways in which we can implement a behavioral testing framework specifically for React components. As we mature in our ability to identify test cases, write tests, and avoid regressions, we’ll follow up with another blog post detailing our findings.

Have designs for either static HTML or a loading screen for larger, more complex components.

Because ReactJS replaces elements of the DOM after the page has loaded, there always tends to be a brief flash where an empty div is suddenly populated and pushes content down the page. For larger components (either in terms of dimensions or complexity), it can be a jarring experience.

If you have a dedicated designer for your project, I recommend designing either static HTML that will be replaced and styling it with at least min-height to mitigate that “jump;” or design a loading animation that will display until the container div is replaced by React.

If you’re progressively decoupling a component, use it everywhere. (Don’t mix Twig and React!)

Thankfully, we identified this early on in the development process. To make the student story and opportunity card components, we first built the component in static HTML (via Twig) and CSS; I then converted the Twig templates into JSX. Later on, I discovered that we were displaying student story cards using that Twig template, not a React component. We did some quick rewriting of the pages where we would have placed static story components to use React components, because we wanted to avoid updating components in two different places if we ever needed to change the markup for the story component. Now, we have Twig templates for non-React components, and JSX templates for React components, with no overlap.

Start earlier than you think you need to.

This is less of a lesson specifically related to decoupling Drupal and more of a general lesson in project management: If you can start something, start it. On this project, an over-simplified estimate of the level of effort led us to defer the start of the React work until the final third of the project, which led to some stress and longer hours as we went into our refactoring phase. As we continue to add tools into our development toolbox, it will be important for us to remember that there’s usually nothing to lose and plenty to gain by starting larger pieces of functionality earlier in the project.

Going Forward

As we head into an internal project retrospective, I’m sure we’ll discover that there are things that we could have done better, or would have simply done differently. But that’s probably true of every project that any developer has ever worked on. At the end of the day, we’re very proud of the work that we did, and I’m happy that we have a project that we can refer to next time we encounter something on a new design that looks like a good candidate for a React component. And while at the time, it was concerning to me that so many developers touched the project, I realized that now fully half of my team has some real-world experience creating a progressively decoupled Drupal project, instead of just one developer.

I invite you to head over to https://admissions.upenn.edu and check out the work that my team and our designers did in bringing this site to life. When you’re ready to look at the finished React work, go to https://admissions.upenn.edu/explore-interests, select some interests, and browse around the site.