Learning Design and Monitoring in CSCL Scenarios: Emergent Synergies

Background
Day by day, learning scenarios are gaining complexity with the combination of individual, group and whole-class tasks as well as face-to-face or distance activities. Furthermore, the integration of a wide range of technologies such as virtual and personal learning environments (VLEs/PLEs), web 2.0 or mobile tools is becoming increasingly common. These changes have made the management of technological-enhanced classrooms highly demanding. There are many tasks to be carried out by teachers to orchestrate [1] a real classroom, such as intervening at any time to change activities, modifying the deadlines, re-structuring groups, etc. Besides, the number and complexity of these tasks increase when active pedagogies are used, such as in computer-supported collaborative learning (CSCL) or Inquiry-Based Learning (IBL) scenarios.

Since orchestration implies making interventions in response to the specific context and emergent eventualities, one crucial aspect is to provide awareness tools that show what is happening in the learning scenario. For this purpose, monitoring students’ computer-mediated interactions may be very helpful in order to show what is “hidden” behind the technology.

Considerable efforts have been done in order to support CSCL scenarios, developing tools for design, enactment and evaluation purposes. For instance, scripting and monitoring are two long discussed techniques in the research community aimed to foster effective collaboration [2]. On the one hand, CSCL scripting structures the learning scenario and provides students with a set of instructions that guide potentially fruitful collaboration; on the other hand, monitoring the collaboration facilitates the intervention of the teacher in order to redirect the group work in a more productive direction.

Linking Pedagogical Intentions with Monitoring Support
Previous research has pointed out that synergies may appear when monitoring and learning design are aligned [3]. Learning design would benefit from taking into account the special requirements posed by monitoring, and, moreover, the integration during the monitoring of reminders on issues from the design could help to obtain results better tailored to the teacher’s needs.

Thus, even though it is one important factor in building effective collaborative learning, having a “plan” is not enough to ensure that the desired collaboration occurs:  eventualities and contingencies require modifications of the course of the learning  situation, leading teachers to adapt the original design. At this point, monitoring the  learning scenario and comparing its actual and expected states may provide useful information that the teacher can use to regulate collaboration [8].

Conversely, in CSCL research, monitoring has been mostly focused on detailed interaction analysis methods, which provide highly detailed accounts of the collaboration. However, in authentic scenarios, teachers need simpler and easier to interpret feedback so they can react on time if needed. Indeed, the use of diverse ICT tools in the classroom would require participants to integrate monitoring data from all the tools and environments where the learning process takes place. To address these problems, the information from the designed script could be used to guide the data gathering and to contextualize the monitoring results, taking into account elements such as the deadlines, the group structuring and the tools involved in each activity.

Despite the benefits that the combination of scripting and monitoring could offer, no integration has been observed into mainstream CSCL practices. As Martinez et al. explain in [3], the combination is not straightforward. There are several problems that hinder the application of monitoring in real CSCL scenarios. Among these problems, some have a technological origin and some depend on the decisions taken during the design of the learning situation. That lack of attention to the monitoring issues when designing learning scenarios often causes that the technological context is not suitable to be monitored. This reasoning leads us to think that the learning design must take into account the special requirements posed by monitoring at design-time.

Thus, the main goal of this proposal is to provide non-expert teachers with design and enactment tools capable of linking pedagogical intentions with monitoring support for orchestrating blended CSCL scenarios. To achieve this goal, we have defined three partial objectives:
1. Support non-expert teachers to identify and include the monitoring issues of pedagogical significance throughout the design process of CSCL scenarios.
2. Provide teachers with coarse-grained information about the evolution of the CSCL scenario, based on the constraints obtained from the learning design.
3. Support the automation of the data gathering and integration tasks in technologically distributed and heterogeneous CSCL environments.

In fact, although our proposal focuses more on CSCL situations, we believe that similar benefits could be obtained from applying it to teaching in IBL scenarios.

Viability: Preliminary Results

Several tasks have been already done in relation to each one of the objectives mentioned in the previous section, and multiple exploratory studies in authentic learning scenarios have taken place so far. In [7], we found initial evidence that being aware of the pattern used in a script increases the opportunities of detecting critical situations when the script is put into practice. This way, monitoring process is more efficient, since the critical points are identified in advance, and monitoring can focus on detecting them, instead of on modeling the whole collaborative process.

In [4] we delved into the problem of data gathering in DLEs for monitoring purposes. A solution was proposed to add monitoring functionalities to an existing architecture devoted to integrate virtual and personal learning environments (VLEs and PLEs) with external tools, named GLUE! (Group Learning Uniform Environment). Initial evidence was obtained on the capabilities of the proposed architecture to gather relevant information about the users’ actions during the learning process.

Currently, our efforts are focused on the formalization of both the learning design and  monitoring process, and the monitoring model. The first steps done in this direction may be found in [6] [5]. We expect to provide further insights and evaluation data on these topics during the workshop.

References
1. P. Dillenbourg, S. J¨arvel¨a, and F. Fischer. The evolution of research on computer-supported collaborative learning. In Nicolas Balacheff, Sten Ludvigsen, Ton Jong, Ard Lazonder, and Sally Barnes, editors, Technology-Enhanced Learning, pages 3–19. Springer Netherlands, 2009.
2. P. Jermann, A. Soller, and A. Lesgold. Computer Software Support for CSCL. In J.W Strijbos, P. Kirschner, and R. Martens, editors, What we know about CSCL ... and  implementing it in higher education, volume 3 of Computer-Supported Collaborative Learning, pages 141–166. Kluwer Academic Publishers, Norwell, MA, USA, 2004.
3. A. Martínez-Monés, Y. Dimitriadis, and A. Harrer. An interaction aware design process for the integration of interaction analysis in mainstream CSCL practices. In S. Puntambekar, G. Erkens, and C. Hmelo-Silver, editors, Analyzing Interactions in CSCL: Methods, Approaches and Issues, pages 269–291. Springer US, 2011.
4. M.J. Rodríguez-Triana, A. Martínez-Monés, J.I. Asensio-Pérez. Monitoring Collaboration in Flexible and Personal Learning Environments. Interaction, Design and Architecture(s) Journal, special issue on: Evaluating Educative Experiences of Flexible and Personal Learning Environments., 11(12):51–63, 2011.
5. M.J. Rodríguez-Triana, A. Martínez-Monés, J.I. Asensio-Pérez, and Y. Dimitriadis. Script-Aware Monitoring Model: Using Teachers’ Pedagogical Intentions to Guide Learning Analytics. In Towards Theory and Practice of Teaching Analytics (TAPTA) Workshop, EC-TEL’12, Saarbr¨ucken (Germany), 2012. 7th European Conference on Technology Enhanced Learning: 21st Century Learning for 21st Century Skills.
6. M.J. Rodríguez-Triana, A. Martínez-Monés, J.I. Asensio-Pérez, and Y. Dimitriadis. Towards a monitoring-aware learning design process. In 18th International Conference on Collaboration and Technology, CRIWG’12, Duisburg, Germany, 2012.
7. M.J. Rodríguez-Triana, A. Martínez-Monés, J.I. Asensio-Pérez, I.M. Jorrín-Abellán, and Y.
Dimitriadis. Monitoring pattern-based CSCL scripts: a case study. In Proceedings of the
6th European Conference on Technology Enhanced Learning: Towards Ubiquitous Learning, EC-TEL’11, Palermo, Italy, 2011.
8. A. Soller, A. Martínez, P. Jermann, and M. Muehlenbrock. From Mirroring to Guiding: a review of the state of the art in interaction analysis. International Journal on Artificial Intelligence in Education, 15:261–290, 2005.