Next level Shiny- R, Python, and JavaScript

# Next level Shiny- R, Python, and JavaScript
----
## **Shiny in Production conference, 7th October 2022**
### Chris Beeley | October 2022

---

# Introduction

* A history of Shiny
* What does Shiny do well? 
* What does Shiny not do well?
* How can we combine Shiny with other tools to improve our applications?

---

# Shiny - a history

* This is my personal recollection. YMMV
* In the early days it was really more suitable for exploring your own data
* Lacked good documentation, modules and many important functions
* Lots of self taught programmers writing applications with little good practice guidance
* With predictable consequences 🙃

---

# When all you have is a hammer...

* (everything looks like a nail)
* If you only know Excel...
    * You end up writing complex fragile models in it
* If you're a data scientist that only knows Shiny for interaction...
    * You end up writing all of your interactive code in R

---

# Data science is a team sport

* Very often we see data scientists building Shiny applications of their own models
* This is fine for testing, not so fine for production
* The world needs Shiny developers

---

# What are the skills of a Shiny developer?

* (some of...)
    * R/ Shiny 😉
    * DevOps, deployment, {pins}, {targets}...
    * Web development, JavaScript, accessibility, visualisation
    * Working with modules and working with codebases written by others
    * Using internal packages for data processing and analysis
        * This code could be in R or Python

---

# The context

* The NHS collects a lot of patient experience data
* Rate the service 1-5 (Very poor... Excellent) but also give written feedback
    * "Parking was difficult"
    * "Doctor was rude"
    * "You saved my life"
* Many organisations lack the staffing to read all of the feedback in a systematic way
* We have built an algorithm to read it
    * Theme
    * "Criticality"

---

# Patient experience 101

* Tick box scoring is not useful (or accurate)
* Text based data is _complex_ and built on _human experience_
* We're not making word clouds!
* We're not classifying movie reviews or Reddit posts
* The tool should enhance, not replace, human understanding
* "A recommendation engine for feedback data"

---

# The work

* We have had a year of funding to build a text classification model for patient feedback
* We now have another year of funding to make it better, especially the frontend
* The model itself is written in Python
* Marked up and classified text data is a very rich source of data
* We need to build quite a sophisticated user interface to get the best out of this data

---

# The challenge

* We need...
* Performant, powerful, state of the art text mining and ML
* A highly interactive user interface which maximises the information we give to the user and maximises the information we collect from the enduser
* I think it's probably fair to say that R/ Shiny is not particularly good at either

---

# To the rescue!

---

# Python! Huh! What is it good for?

* Absolutely machine learning!
* The project makes (or will make) free use of loads of Python packages, including:
  * Scikit-learn
  * Gensim
  * Text blob
  * Spacy
  * Vader
* Python ML is fast and multicore out of the box

---

# Well how do you do it?

* [Demo time!](https://involve.nottshc.nhs.uk/rsconnect/jane_eyre/)

---

# So how _do_ you do it?

* With {reticulate} (natch)
* With functions
* That's it!

```python
from textblob import TextBlob

def sentiment(text):
  text_blob = TextBlob(text)
  return(text_blob.sentiment.polarity)
```

---

# And over in your Shiny application...

```r
# load functions
reticulate::source_python("python_code.py")

output$histogram <- renderPlot({
  
  sentiment_score <- purrr::map_dbl(selected_text(), function(x){
    sentiment(x)
  })
  
  df_graph <- data.frame("sentiment" = sentiment_score)
  
  df_graph |>
    ggplot2::ggplot(ggplot2::aes(sentiment)) +
    ggplot2::geom_histogram() + ggplot2::xlim(-1, 1)
})
```

---

# Wait, there's more...

* [You can import Python packages directly and call them from R](https://rstudio.github.io/reticulate/#importing-python-modules)
* [You can run Python scripts as files or directly as quoted code and access the objects they create as R objects](https://rstudio.github.io/reticulate/articles/calling_python.html#executing-code)
* [You can build an R wrapper to your Python code and just call that directly](https://rstudio.github.io/reticulate/articles/package.html)

---

# And now to JavaScript

* Every time we interact with a Shiny application we hit the server
* There are lots of ways of limiting and otherwise handling our interactions with the server
    * Reactive functions
    * Caching outputs and other intermediate steps
    * Throttling, debouncing
    * ... you're at a Shiny conference. You know a lot of this
    
---

# There is another way

* What if we could get the user's computer to do some work?
* It's probably pretty much idle while they're using our app, so let's borrow the CPU to do some stuff for them
    * Not crypto mining (that's another talk!)
    * We can run JavaScript in the browser
* There is lots of JavaScript packaged up for R
    * The DT package uses JavaScript
    * {shinyjs} has LOADS of useful JavaScript in it

---

# All your JavaScript are belong to us

* You can take it further without really too much effort
    * You can use JavaScript packages
    * You can write your own JavaScript
* [Demo time!](https://involve.nottshc.nhs.uk/rsconnect/jane_eyre/)

---

# Golem to the rescue!

* Golem even makes this easy

```r
golem::add_js_file("heart")
golem::add_css_file("heart")
```

---

# My heart will go in inst/

* JavaScript and any modules that are called go in inst/app/www
* Any files the browser needs like graphics go in there too
    * (and are referred to in code just as "www/filename.png")

---

# Why are we doing this again?

* There are non silly applications of this kind of work
* We can mark up and highlight text in complex ways
    * We can mark negative sentences
    * We can mark words on a theme (care words, staff words, emotional words...)
    * We can use a huge range of JavaScript output widgets to make interesting visualisations
    * Devise methods of collecting data from the user
        * _That_ sentence is about *this*, _this_ sentence is about *that*

---

# That's all, folks!

* Let Shiny do what Shiny does best
* Don't be afraid to bring in other languages and packages in other languages to do what they do best
* Train, recruit, and retain Shiny developers
    * Allow them to develop and use a diverse skillset

---

# Me

[<svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"></path></svg> @ChrisBeeley](https://twitter.com/ChrisBeeley)

[<svg viewBox="0 0 496 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"></path></svg> My GitHub](https://github.com/ChrisBeeley)

## The team

[<svg viewBox="0 0 512 512" style="height:1em;position:relative;display:inline-block;top:.1em;" xmlns="http://www.w3.org/2000/svg">  <path d="M440 6.5L24 246.4c-34.4 19.9-31.1 70.8 5.7 85.9L144 379.6V464c0 46.4 59.2 65.5 86.6 28.6l43.8-59.1 111.9 46.2c5.9 2.4 12.1 3.6 18.3 3.6 8.2 0 16.3-2.1 23.6-6.2 12.8-7.2 21.6-20 23.9-34.5l59.4-387.2c6.1-40.1-36.9-68.8-71.5-48.9zM192 464v-64.6l36.6 15.1L192 464zm212.6-28.7l-153.8-63.5L391 169.5c10.7-15.5-9.5-33.5-23.7-21.2L155.8 332.6 48 288 464 48l-59.4 387.3z"></path></svg> cdudatascience@nottshc.nhs.uk](mailto:cdudatascience@nottshc.nhs.uk)